Featureitis

You don’t have to be a software developer to recognize a good application. But from my own experience, I’ve often seen programs that were promising and innovative at the start mutate into unwieldy behemoths once they reach a certain number of users. Since I’ve been making this observation regularly for several decades now, I’ve wondered what the reasons for this might be.

The phenomenon of software programs, or solutions in general, becoming overloaded with details was termed “featuritis” by Brooks in his classic book, “The Mythical Man-Month.” Considering that the first edition of the book was published in 1975, it’s fair to say that this is a long-standing problem. Perhaps the most famous example of featureitis is Microsoft’s Windows operating system. Of course, there are countless other examples of improvements that make things worse.

The phenomenon of software programs, or solutions in general, becoming overloaded with details is what Brooks called “featuritis.” Windows users who were already familiar with Windows XP and then confronted with its wonderful successor Vista, only to be appeased again by Windows 7, and then nearly had a heart attack with Windows 8 and 8.1, were calmed down again at the beginning of Windows 10. At least for a short time, until the forced updates quickly brought them back down to earth. And don’t even get me started on Windows 11. The old saying about Windows was that every other version is junk and should be skipped. Well, that hasn’t been true since Windows 7. For me, Windows 10 was the deciding factor in completely abandoning Microsoft, and like many others, I bought a new operating system. Some switched to Apple, and those who couldn’t afford or didn’t want the expensive hardware, like me, opted for a Linux system. This shows how a lack of insight can quickly lead to the loss of significant market share. Since Microsoft isn’t drawing any conclusions from these developments, this fact seems to be of little concern to the company. For other companies, such events can quickly push them to the brink of collapse, and beyond.

One motivation for adding more and more features to an existing application is the so-called product life cycle, which is represented by the BCG matrix in Figure 1.

With a product’s launch, it’s not yet certain whether it will be accepted by the market. If users embrace it, it quickly rises to stardom and reaches its maximum market position as a cash cow. Once market saturation is reached, it degrades to a slow seller. So far, so good. Unfortunately, the prevailing management view is that if no growth is generated compared to the previous quarter, market saturation has already been reached. This leads to the nonsensical assumption that users must be forced to accept an updated version of the product every year. Of course, the only way to motivate a purchase is to print a bulging list of new features on the packaging.

Since well-designed features can’t simply be churned out on an assembly line, a redesign of the graphical user interface is thrown in as a free bonus every time. Ultimately, this gives the impression of having something completely new, as it requires a period of adjustment to discover the new placement of familiar functions. It’s not as if the redesign actually streamlines the user experience or increases productivity. The arrangement of input fields and buttons always seems haphazardly thrown together.

But don’t worry, I’m not calling for an update boycott; I just want to talk about how things can be improved. Because one thing is certain: thanks to artificial intelligence, the market for software products will change dramatically in just a few years. I don’t expect complex and specialized applications to be produced by AI algorithms anytime soon. However, I do expect that these applications will have enough poorly generated AI-generated code sequences, which the developer doesn’t understand, injected into their codebases, leading to unstable applications. This is why I’m rethinking clean, handcrafted, efficient, and reliable software, because I’m sure there will always be a market for it.

I simply don’t want an internet browser that has mutated into a communication hub, offering chat, email, cryptocurrency payments, and who knows what else, in addition to simply displaying web pages. I want my browser to start quickly when I click something, then respond quickly and display website content correctly and promptly. If I ever want to do something else with my browser, it would be nice if I could actively enable this through a plugin.

Now, regarding the problem just described, the argument is often made that the many features are intended to reach a broad user base. Especially if an application has all possible options enabled from the start, it quickly engages inexperienced users who don’t have to first figure out how the program actually works. I can certainly understand this reasoning. It’s perfectly fine for a manufacturer to focus exclusively on inexperienced users. However, there is a middle ground that considers all user groups equally. This solution isn’t new and is very well-known: the so-called product lines.

In the past, manufacturers always defined target groups such as private individuals, businesses, and experts. These user groups were then often assigned product names like Home, Enterprise, and Ultimate. This led to everyone wanting the Ultimate version. This phenomenon is called Fear Of Missing Out (FOMO). Therefore, the names of the product groups and their assigned features are psychologically poorly chosen. So, how can this be done better?

An expert focuses their work on specific core functions that allow them to complete tasks quickly and without distractions. For me, this implies product lines like Essentials, Pure, or Core.

If the product is then intended for use by multiple people within the company, it often requires additional features such as external user management like LDAP or IAM. This specialized product line is associated with terms like Enterprise, Company, Business, and so on.

The cluttered end result, actually intended for NOOPS, has all sorts of things already activated during installation. If people don’t care about the application’s startup and response time, then go for it. Go all out. Throw in everything you can! Here, names like Ultimate, Full, and Maximized Extended are suitable for labeling the product line. The only important thing is that professionals recognize this as the cluttered version.

Those who cleverly manage these product lines and provide as many functions as possible via so-called modules, which can be installed later, enable high flexibility even in expert mode, where users might appreciate the occasional additional feature.

If you install tracking on the module system beforehand to determine how professional users upgrade their version, you’ll already have a good idea of ​​what could be added to the new version of Essentials. However, you shouldn’t rely solely on downloads as the decision criterion for this tracking. I often try things out myself and delete extensions faster than the installation process took if I think they’re useless.

I’d like to give a small example from the DevOps field to illustrate the problem I just described. There’s the well-known GitLab, which was originally a pure code repository hosting project. The name still reflects this today. An application that requires 8 GB of RAM on a server in its basic installation just to make a Git repository accessible to other developers is unusable for me, because this software has become a jack-of-all-trades over time. Slow, inflexible, and cluttered with all sorts of unnecessary features that are better implemented using specialized solutions.

In contrast to GitLab, there’s another, less well-known solution called SCM-Manager, which focuses exclusively on managing code repositories. I personally use and recommend SCM-Manager because its basic installation is extremely compact. Despite this, it offers a vast array of features that can be added via plugins.

I tend to be suspicious of solutions that claim to be an all-in-one solution. To me, that’s always the same: trying to do everything and nothing. There’s no such thing as a jack-of-all-trades, or as we say in Austria, a miracle worker!

When selecting programs for my workflow, I focus solely on their core functionality. Are the basic features promised by the marketing truly present and as intuitive as possible? Is there comprehensive documentation that goes beyond a simple “Hello World”? Does the developer focus on continuously optimizing core functions and consider new, innovative concepts? These are the questions that matter to me.

Especially in commercial environments, programs are often used that don’t deliver on their marketing promises. Instead of choosing what’s actually needed to complete tasks, companies opt for applications whose descriptions are crammed with buzzwords. That’s why I believe that companies that refocus on their core competencies and use highly specialized applications for them will be the winners of tomorrow.


Mismanagement and Alpha Geeks

When I recently picked up Camille Fournier’s book “The Manager’s Path,” I was immediately reminded of Tom DeMarco. He wrote the classic “Peopleware” and, in the early 2000s, published “Adrenaline Junkies and Form Junkies.” It’s a list of stereotypes you might encounter in software projects, with advice on how to deal with them. After several decades in the business, I can confirm every single word from my own experience. And it’s still relevant today, because people are the ones who make projects, and we all have our quirks.

For projects to be successful, it’s not just technical challenges that need to be overcome. Interpersonal relationships also play a crucial role. One important factor in this context, which often receives little attention, is project management. There are shelves full of excellent literature on how to become a good manager. The problem, unfortunately, is that few who hold this position actually fulfill it, and even fewer are interested in developing their skills further. The result of poor management is worn-down and stressed teams, extreme pressure in daily operations, and often also delayed delivery dates. It’s no wonder, then, that this impacts product quality.

One of the first sayings I learned in my professional life was: “Anyone who thinks a project manager actually manages projects also thinks a butterfly folds lemons.” So it seems to be a very old piece of wisdom. But what is the real problem with poor management? Anyone who has to fill a managerial position has a duty to thoroughly examine the candidate’s skills and suitability. It’s easy to be impressed by empty phrases and a list of big names in the industry on a CV, without questioning actual performance. In my experience, I’ve primarily encountered project managers who often lacked the necessary technical expertise to make important decisions. It wasn’t uncommon for managers in IT projects to dismiss me with the words, “I’m not a technician, sort this out amongst yourselves.” This is obviously disastrous when the person who’s supposed to make the decisions can’t make them because they lack the necessary knowledge. An IT project manager doesn’t need to know which algorithm will terminate the project faster. Evaluations can be used to inform decisions. However, a basic understanding of programming is essential. Anyone who doesn’t know what an API is and why version compatibility prevents modules that will later be combined into a software product from working together has no right to act as a decision-maker. A fundamental understanding of software development processes and the programming paradigms used is also indispensable for project managers who don’t work directly with the code.

I therefore advocate for vetting not only the developers you hire for their skills, but also the managers who are to be brought into a company. For me, external project management is an absolute no-go when selecting my projects. This almost always leads to chaos and frustration for everyone involved, which is why I reject such projects. Managers who are not integrated into the company and whose performance is evaluated based on project success, in my experience, do not deliver high-quality work. Furthermore, internal managers, just like developers, can develop and expand their skills through mentoring, training, and workshops. The result is a healthy, relaxed working environment and successful projects.

The title of this article points to toxic stereotypes in the project business. I’m sure everyone has encountered one or more of these stereotypes in their professional environment. There’s a lot of discussion about how to deal with these individuals. However, I would like to point out that hardly anyone is born a “monster.” People are the way they are, a result of their experiences. If a colleague learns that looking stressed and constantly rushed makes them appear more productive, they will perfect this behavior over time.

Camille Fournier aptly described this with the term “The Alpha Geek.” Someone who has made their role in the project indispensable and has an answer for everything. They usually look down on their colleagues with disdain, but can never truly complete a task without others having to redo it. Unrealistic estimates for extensive tasks are just as typical as downplaying complex issues. Naturally, this is the darling of all project managers who wish their entire team consisted of these “Alpha Geeks.” I’m quite certain that if this dream could come true, it would be the ultimate punishment for the project managers who create such individuals in the first place.

To avoid cultivating “alpha geeks” within your company, it’s essential to prevent personality cults and avoid elevating personal favorites above the rest of the team. Naturally, it’s also crucial to constantly review work results. Anyone who marks a task as completed but requires rework should be reassigned until the result is satisfactory.

Personally, I share Tom DeMarco’s view on the dynamics of a project. While productivity can certainly be measured by the number of tasks completed, other factors also play a crucial role. My experience has taught me that, as mentioned earlier, it’s essential to ensure employees complete all tasks thoroughly and thoroughly before taking on new ones. Colleagues who downplay a task or offer unrealistic, low-level assessments should be assigned that very task. Furthermore, there are colleagues who, while having relatively low output, contribute significantly to team harmony.

When I talk about people who build a healthy team, I don’t mean those who simply hand out sweets every day. I’m referring to those who possess valuable skills and mentor their colleagues. These individuals typically enjoy a high level of trust within the team, which is why they often achieve excellent results as mediators in conflicts. It’s not the people who try to be everyone’s darling with false promises, but rather those who listen and take the time to find a solution. They are often the go-to people for everything and frequently have a quiet, unassuming demeanor. Because they have solutions and often lend a helping hand, they themselves receive only average performance ratings in typical process metrics. A good manager quickly recognizes these individuals because they are generally reliable. They are balanced and appear less stressed because they proceed calmly and consistently.

Of course, much more could be said about the stereotypes in a software project, but I think the points already made provide a good basic understanding of what I want to express. An experienced project manager can address many of the problems described as they arise. This naturally requires solid technical knowledge and some interpersonal skills.

Of course, we must also be aware that experienced project managers don’t just appear out of thin air. They need to be developed and supported, just like any other team member. This definitely includes rotations through all technical departments, such as development, testing, and operations. Paradigms like pair programming are excellent for this. The goal isn’t to turn a manager into a programmer or tester, but rather to give them an understanding of the daily processes. This also strengthens confidence in the skills of the entire team, and mentalities like “you have to control and push lazy and incompetent programmers to get them to lift a finger” don’t even arise. In projects that consistently deliver high quality and meet their deadlines, there’s rarely a desire to introduce every conceivable process metric.


Blockchain – an introduction

The blockchain concept is a fundamental component of various crypto payment methods such as Bitcoin and Ethereum. But what exactly is blockchain technology, and what other applications does this concept have? Essentially, blockchain is structured like a backward-linked list. Each element in the list points to its predecessor. So, what makes blockchain so special?

Blockchain extends the list concept by adding various constraints. One of these constraints is ensuring that no element in the list can be altered or removed. This is relatively easy to achieve using a hash function. We encode the content of each element in the list into a hash using a hash algorithm. A wide range of hash functions are now available, with SHA-512 being a current standard. This hash algorithm is already implemented in the standard library of almost every programming language and can be used easily. Specifically, this means that the SHA-512 hash is generated from all the data in a block. This hash is always unique and never occurs again. Thus, the hash serves as an identifier (ID) to locate a block. An entry in the block is a reference to its predecessors. This reference is the hash value of the predecessor, i.e., its ID. When implementing a blockchain, it is essential that the hash value of the predecessor is included in the calculation of the hash value of the current block. This detail ensures that elements in the blockchain can only be modified with great difficulty. Essentially, to manipulate the element one wishes to alter, all subsequent elements must also be changed. In a large blockchain with a very large number of blocks, such an undertaking entails an enormous computational effort that is very difficult, if not impossible, to accomplish.

This chaining provides us with a complete transaction history. This also explains why crypto payment methods are not anonymous. Even though the effort required to uniquely identify a transaction participant can be enormous, if this participant also uses various obfuscation methods with different wallets that are not linked by other transactions, the effort increases exponentially.

Of course, the mechanism just described still has significant weaknesses. Transactions, i.e., the addition of new blocks, can only be considered verified and secure once enough successors have been added to the blockchain to ensure that changes are more difficult to implement. For Bitcoin and similar cryptocurrencies, a transaction is considered secure if there are five subsequent transactions.

To avoid having just one entity storing the transaction history—that is, all the blocks of the blockchain—a decentralized approach comes into play. This means there is no central server acting as an intermediary. Such a central server could be manipulated by its operator. With sufficient computing power, this would allow for the rebuilding of even very large blockchains. In the context of cryptocurrencies, this is referred to as chain reorganization. This is also the criticism leveled at many cryptocurrencies. Apart from Bitcoin, no other decentralized and independent cryptocurrency exists. If the blockchain, with all its contained elements, is made public and each user has their own instance of this blockchain locally on their computer, where they can add elements that are then synchronized with all other instances of the blockchain, then we have a decentralized approach.

The technology for decentralized communication without an intermediary is called peer-to-peer (P2P). P2P networks are particularly vulnerable in their early stages, when there are only a few participants. With a great deal of computing power, one could easily create a large number of so-called “Zomi peers” that influence the network’s behavior. Especially in times when cloud computing, with providers like AWS and Google Cloud Platform, can provide virtually unlimited resources for relatively little money, this is a significant problem. This point should not be overlooked, particularly when there is a high financial incentive for fraudsters.

There are also various competing concepts within P2P. To implement a stable and secure blockchain, it is necessary to use only solutions that do not require supporting backbone servers. The goal is to prevent the establishment of a master chain. Therefore, questions must be answered regarding how individual peers can find each other and which protocol they use to synchronize their data. By protocol, we mean a set of rules, a fixed framework for how interaction between peers is regulated. Since this point is already quite extensive, I refer you to my 2022 presentation for an introduction to the topic.

Another feature of blockchain blocks is that their validity can be easily and quickly verified. This simply requires generating the SHA-512 hash of the entire contents of a block. If this hash matches the block’s ID, the block is valid. Time-sensitive or time-critical transactions, such as those relevant to payment systems, can also be created with minimal effort. No complex time servers are needed as intermediaries. Each block is appended with a timestamp. This timestamp must, however, take into account the location where it is created, i.e., specify the time zone. To obscure the location of the transaction participants, all times in the different time zones can be converted to the current UTC 0.

To ensure that the time is correctly set on the system, a time server can be queried for the current time when the software starts, and a correction message can be displayed if there are any discrepancies.

Of course, time-critical transactions are subject to a number of challenges. It must be ensured that a transaction was carried out within a defined time window. This is a problem that so-called real-time systems have to deal with. The double-spending problem also needs to be prevented—that is, the same amount being sent twice to different recipients. In a decentralized network, this requires confirmation of the transaction by multiple participants. Classic race conditions can also pose a problem. Race conditions can be managed by applying the Immutable design pattern to the block elements.

To prevent the blockchain from being disrupted by spam attacks, we need a solution that makes creating a single block expensive. We achieve this by incorporating computing power. The participant creating a block must solve a puzzle that requires a certain amount of computing time. If a spammer wants to flood the network with many blocks, their computing power increases exorbitantly, making it impossible for them to generate an unlimited number of blocks in a short time. This cryptographic puzzle is called a nonce, which stands for “number used only once.” The nonce mechanism in the blockchain is also often referred to as Proof of Work (POW) and is used in Bitcoin to verify the blocks by the miners.

The nonce is a (pseudo)random number for which a hash must be generated. This hash must then meet certain criteria. These could be, for example, two or three leading zeros in the hash. To prevent arbitrary hashes from being inserted into the block, the random number that solves the puzzle is stored directly. A nonce that has already been used cannot be used again, as this would circumvent the puzzle. When generating the hash from the nonce, it must meet the requirements, such as leading zeros, to be accepted.

Since finding a valid nonce becomes increasingly difficult as the number of blocks in a blockchain grows, it is necessary to change the rules for such a nonce cyclically, for example, every 2048 blocks. This also means that the rules for a valid nonce must be assigned to the corresponding blocks. Such a set of rules for the nonce can easily be formulated using a regular expression (regex).

We’ve now learned a considerable amount about the ruleset for a blockchain. So it’s time to consider performance. If we were to simply store all the individual blocks of the blockchain in a list, we would quickly run out of memory. While it’s possible to store the blocks in a local database, this would negatively impact the blockchain’s speed, even with an embedded solution like SQLite. A simple solution would be to divide the blockchain into equal parts, called chunks. A chunk would have a fixed length of 2048 valid blocks, and the first block of a new chunk would point to the last block of the previous chunk. Each chunk could also contain a central rule for the nonce and store metadata such as minimum and maximum timestamps.

To briefly recap our current understanding of the blockchain ruleset, we’re looking at three different levels. The largest level is the blockchain itself, which contains fundamental metadata and configurations. Such configurations include the hash algorithm used. The second level consists of so-called chunks, which contain a defined set of block elements. As mentioned earlier, chunks also contain metadata and configurations. The smallest element of the blockchain is the block itself, which comprises an ID, the described additional information such as a timestamp and nonce, and the payload. The payload is a general term for any data object that is to be made verifiable by the blockchain. For Bitcoin and other cryptocurrencies, the payload is the information about the amount being transferred from Wallet A (source) to Wallet B (destination).

Blockchain technology is also suitable for many other application scenarios. For example, the hash values ​​of open-source software artifacts could be stored in a blockchain. This would allow users to download binary files from untrusted sources and verify them against the corresponding blockchain. The same principle could be applied to the signatures of antivirus programs. Applications and other documents could also be transmitted securely in governmental settings. The blockchain would function as a kind of “postal stamp.” Accounting, including all receipts for goods and services purchased and sold, is another conceivable application.

Depending on the use case, an extension of the blockchain would be the unique signing of a block by its creator. This would utilize the classic PKI (Public Key Infrastructure) method with public and private keys. The signer stores their public key in the block and creates a signature using their private key via the payload, which is then also stored in the block.

Currently, there are two freely available blockchain implementations: BitcoinJ and Web3j for Ethereum. Of course, it’s possible to create your own universally applicable blockchain implementation using the principles just described. The pitfalls, naturally, lie in the details, some of which I’ve already touched upon in this article. Fundamentally, however, blockchain isn’t rocket science and is quite manageable for experienced developers. Anyone considering trying their hand at their own implementation now has sufficient basic knowledge to delve deeper into the necessary details of the various technologies involved.


PDF generation with TP-CORE

PDF is arguably the most important interoperable document format between operating systems and devices such as computers, smartphones, and tablets. The most important characteristic of PDF is the immutability of the documents. Of course, there are limits, and nowadays, pages can easily be removed or added to a PDF document using appropriate programs. However, the content of individual pages cannot be easily modified. Where years ago expensive specialized software such as Adobe Acrobat was required to work with PDFs, many free solutions are now available, even for commercial use. For example, the common office suites support exporting documents as PDFs.

Especially in the business world, PDF is an excellent choice for generating invoices or reports. This article focuses on this topic. I describe how simple PDF documents can be used for invoicing, etc. Complex layouts, such as those used for magazines and journals, are not covered in this article.

Technically, the freely available library openPDF is used. The concept is to generate a PDF from valid HTML code contained in a template. Since we’re working in the Java environment, Velocity is the template engine of choice. TP-CORE utilizes all these dependencies in its PdfRenderer functionality, which is implemented as a facade. This is intended to encapsulate the complexity of PDF generation within the functionality and facilitate library replacement.

While I’m not a PDF generation specialist, I’ve gained considerable experience over the years, particularly with this functionality, in terms of software project maintainability. I even gave a conference presentation on this topic. Many years ago, when I decided to support PDF in TP-CORE, the only available library was iText, which at that time was freely available with version 5. Similar to Linus Torvalds with his original source control management system, I found myself in a similar situation. iText became commercial, and I needed a new solution. Well, I’m not Linus, who can just whip up Git overnight, so after some waiting, I discovered openPDF. OpenPDF is a fork of iText 5. Now I had to adapt my existing code accordingly. This took some time, but thanks to my encapsulation, it was a manageable task. However, during this adaptation process, I discovered problems that were already causing me difficulties in my small environment, so I released TP-CORE version 3.0 to achieve functional stability. Therefore, anyone using TP-CORE version 2.x will still find iText5 as the PDF solution. But that’s enough about the development history. Let’s look at how we can currently generate PDFs with TP-CORE 3.1. Here, too, my main goal is to achieve the highest possible compatibility.

Before we can begin, we need to include TP-CORE as a dependency in our Java project. This example demonstrates the use of Maven as the build tool, but it can easily be switched to Gradle.

<dependency>
    <groupId>io.github.together.modules</groupId>
    <artifactId>core</artifactId>
    <version>3.1.0</version>
</dependency>

To generate an invoice, for example, we need several steps. First, we need an HTML template, which we create using Velocity. The template can, of course, contain placeholders for names and addresses, enabling mail merge and batch processing. The following code demonstrates how to work with this.

<h1>HTML to PDF Velocity Template</h1>

<p>Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet.</p>

<p>Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit augue duis dolore te feugait nulla facilisi. Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat.</p>

<h2>Lists</h2>
<ul>
    <li><b>bold</b></li>
    <li><i>italic</i></li>
    <li><u>underline</u></li>
</ul>

<ol>
    <li>Item</li>
    <li>Item</li>
</ol>

<h2>Table 100%</h2>
<table border="1" style="width: 100%; border: 1px solid black;">
    <tr>
        <td>Property 1</td>
        <td>$prop_01</td>
        <td>Lorem ipsum dolor sit amet,</td>
    </tr>
    <tr>
        <td>Property 2</td>
        <td>$prop_02</td>
        <td>Lorem ipsum dolor sit amet,</td>
    </tr>
    <tr>
        <td>Property 3</td>
        <td>$prop_03</td>
        <td>Lorem ipsum dolor sit amet,</td>
    </tr>
</table>

<img src="path/to/myImage.png" />

<h2>Links</h2>
<p>here we try a simple <a href="https://together-platform.org/tp-core/">link</a></p>

template.vm

The template contains three properties: $prop_01, $prop_02, and $prop_03, which we need to populate with values. We can do this with a simple HashMap, which we insert using the TemplateRenderer, as the following example shows.

Map<String, String> properties = new HashMap<>();
properties.put("prop_01", "value 01");
properties.put("prop_02", "value 02");
properties.put("prop_03", "value 03");

TemplateRenderer templateEngine = new VelocityRenderer();
String directory = Constraints.SYSTEM_USER_HOME_DIR + "/";
String htmlTemplate = templateEngine
                .loadContentByFileResource(directory, "template.vm", properties);

The TemplateRenderer requires three arguments:

  • directory – The full path where the template is located.
  • template – The name of the Velocity template.
  • properties – The HashMap containing the variables to be replaced.

The result is valid HTML code, from which we can now generate the PDF using the PdfRenderer. A special feature is the constant SYSTEM_USER_HOME_DIR, which points to the user directory of the currently logged-in user and handles differences between operating systems such as Linux and Windows.

PdfRenderer pdf = new OpenPdfRenderer();
pdf.setTitle("Title of my own PDF");
pdf.setAuthor("Elmar Dott");
pdf.setSubject("A little description about the PDF document.");
pdf.setKeywords("pdf, html, openPDF");
pdf.renderDocumentFromHtml(directory + "myOwn.pdf", htmlTemplate);

The code for creating the PDF is clear and quite self-explanatory. It’s also possible to hardcode the HTML. Using the template allows for a separation of code and design, which also makes later adjustments flexible.

The default format is A4 with the dimensions size: 21cm 29.7cm; margin: 20mm 20mm; and is defined as inline CSS. This value can be customized using the setFormat(String format); method. The first value represents the width and the second value the height.

The PDF renderer also allows you to delete individual pages from a document.

PdfRenderer pdf = new OpenPdfRenderer();
PdfDocument original = pdf.loadDocument( new File("document.pdf") );
PdfDocument reduced = pdf.removePage(original, 1, 5);
pdf.writeDocument(reduced, "reduced.pdf");

We can see, therefore, that great emphasis was placed on ease of use when implementing the PDF functionality. Nevertheless, there are many ways to create customized PDF documents. This makes the solution of defining the layout via HTML particularly appealing. Despite the PdfRenderer’s relatively limited functionality, Java’s inheritance mechanism makes it very easy to extend the interface and its implementation with custom solutions. This possibility also exists for all other implementations in TP-CORE.

TP-CORE is also free for commercial use and has no restrictions thanks to the Apache 2 license. The source code can be found at https://git.elmar-dott.com and the documentation, including security and test coverage, can be found at https://together-platform.org/tp-core/.

RTFM – usable documentation

An old master craftsman used to say: “He who writes, remains.” His primary intention was to obtain accurate measurements and weekly reports from his journeymen. He needed this information to issue correct invoices, which was crucial to the success of his business. This analogy can also be readily applied to software development. It wasn’t until Ruby, the programming language developed in Japan by Yukihiro Matsumoto, had English-language documentation that Ruby’s global success began.

We can see, therefore, that documentation can be of considerable importance to the success of a software project. It’s not simply a repository of information within the project where new colleagues can find necessary details. Of course, documentation is a rather tedious subject for developers. It constantly needs to be kept up-to-date, and they often lack the skills to put their own thoughts down on paper in a clear and organized way for others to understand.

I myself first encountered the topic of documentation many years ago through reading the book “Software Engineering” by Johannes Siedersleben. Ed Yourdon was quoted there as saying that before methods like UML, documentation often took the form of a Victorian novella. During my professional life, I’ve also encountered a few such Victorian novellas. The frustrating thing was: after battling through the textual desert—there’s no other way to describe the feeling than as overcoming and struggling—you still didn’t have the information you were looking for. To paraphrase Goethe’s Faust: “So here I stand, poor fool, no wiser than before.”

Here we already see a first criticism of poor documentation: inappropriate length and a lack of information. We must recognize that writing isn’t something everyone is born with. After all, you became a software developer, not a book author. For the concept of “successful documentation,” this means that you shouldn’t force anyone to do it and instead look for team members who have a knack for it. This doesn’t mean, however, that everyone else is exempt from documentation tasks. Their input is essential for quality. Proofreading, pointing out errors, and suggesting additions are all necessary tasks that can easily be shared.

It’s highly advisable to provide occasional rhetorical training for the team or individual team members. The focus should be on precise, concise, and understandable expression. This also involves organizing your thoughts so they can be put down on paper and follow a clear and coherent structure. The resulting improved communication has a very positive impact on development projects.

Up-to-date documentation that is easy to read and contains important information quickly becomes a living document, regardless of the format chosen. This is also a fundamental concept for successful DevOps and agile methodologies. These paradigms rely on good information exchange and address the avoidance of information silos.

One point that really bothers me is the statement: “Our tests are the documentation.” Not all stakeholders can program and are therefore unable to understand the test cases. Furthermore, while tests demonstrate the behavior of functions, they don’t inherently demonstrate their correct usage. Variations of usable solutions are also usually missing. For test cases to have a documentary character, it’s necessary to develop specific tests precisely for this purpose. In my opinion, this approach has two significant advantages. First, the implementation documentation remains up-to-date, because changes will cause the test case to fail. Another positive effect is that the developer becomes aware of how their implementation is being used and can correct a flawed design in a timely manner.

Of course, there are now countless technical solutions that are suitable for different groups of people, depending on their perspective on the system. Issue and bug tracking systems, such as the commercial JIRA or the open-source Redmine, map entire processes. They allow testers to assign identified problems and errors in the software to a specific release version. Project managers can use release management to prioritize fixes, and developers document the implemented fixes. That’s the theory. In practice, I’ve seen in almost every project how the comment function in these systems is misused as a chat to describe the change status. The result is a bug report with countless useless comments, and any real, relevant information is completely missing.

Another widespread technical solution in development projects is the use of enterprise wikis. They enhance simple wikis with navigation and allow the creation of closed spaces where only explicitly authorized user groups receive granular permissions such as read or write. Besides the widely used commercial solution Confluence, there’s also a free alternative called BlueSpice, which is based on MediaWiki. Wikis allow collaborative work on a document, and individual pages can be exported as PDFs in various formats. To ensure that the wiki pages remain usable, it’s important to maintain clean and consistent formatting. Tables should fit their content onto a single A4 page without unwanted line breaks. This improves readability. There are also many instances where bulleted lists are preferable to tables for the sake of clarity.

This brings us to another very sensitive topic: graphics. It’s certainly true that a picture is often worth a thousand words. But not always! When working with graphics, it’s important to be aware that images often require a considerable amount of time to create and can often only be adapted with significant effort. This leads to several conclusions to make life easier. A standard program (format) should be used for creating graphics. Expensive graphics programs like Photoshop and Corel should be avoided. Graphics created for wiki pages should be attached to the wiki page in their original, editable form. A separate repository can also be set up for this purpose to allow reuse in other projects.

If an image offers no added value, it’s best to omit it. Here’s a small example: It’s unnecessary to create a graphic depicting ten stick figures with a role name or person underneath. Here, it is advisable to create a simple list, which is also easier to supplement or adapt.

But you should also avoid overloaded graphics. True to the motto “more is better,” overly detailed information tends to cause confusion and can lead to misinterpretations. A recommended book is “Documenting and Communicating Software Architectures” by Stefan Zörner. In this book, he effectively demonstrates the importance of different perspectives on a system and which groups of people are addressed by a specific viewpoint. I would also like to take this opportunity to share his seven rules for good documentation:

  1. Write from the reader’s perspective.
  2. Avoid unnecessary repetition.
  3. Avoid ambiguity; explain notation if necessary.
  4. Use standards such as UML.
  5. Include the reasons (why).
  6. Keep the documentation up-to-date, but never too up-to-date.
  7. Review the usability.

Anyone tasked with writing the documentation, or ensuring its progress and accuracy, should always be aware that it contains important information and presents it correctly and clearly. Concise and well-organized documentation can be easily adapted and expanded as the project progresses. Adjustments are most successful when the affected area is as cohesive as possible and appears only once. This centralization is achieved through references and hyperlinks, so that changes in the original document are reflected in the references.

Of course, there is much more to say about documentation; it’s the subject of numerous books, but that would go beyond the scope of this article. My main goal was to raise awareness of this topic, as paradigms like Agile and DevOps rely on a good flow of information.


Disk-Jock-Ey II

After discussing more general issues such as file systems and partitions in the first part of this workshop, we’ll turn to various diagnostic techniques in the second and final part of the series. Our primary tool for this will be Bash, as the following tools are all command-line based.

This section, too, requires the utmost care. The practices described here may result in data loss if used improperly. I assume no liability for any damages.

Let’s start with the possibility of finding out how much free space we still have on our hard drive. Anyone who occasionally messes around on servers can’t avoid the df command. After all, the SSH client doesn’t have a graphical interface, and you have to navigate the shell.

With df -hT, all storage, physical and virtual, can be displayed in human-readable format.

ed@local:~$ df -hT
Filesystem     Type      Size  Used Avail Use% Mounted on
udev           devtmpfs   32G     0   32G   0% /dev
tmpfs          tmpfs     6.3G  2.8M  6.3G   1% /run
/dev/nvme0n1p2 ext4      1.8T  122G  1.6T   8% /
tmpfs          tmpfs      32G  8.0K   32G   1% /dev/shm
efivarfs       efivarfs  196K  133K   59K  70% /sys/firmware/efi/efivars
tmpfs          tmpfs     5.0M   16K  5.0M   1% /run/lock
tmpfs          tmpfs     1.0M     0  1.0M   0% /run/credentials/systemd-journald.service
tmpfs          tmpfs      32G   20M   32G   1% /tmp
/dev/nvme0n1p1 vfat      975M  8.8M  966M   1% /boot/efi
tmpfs          tmpfs     6.3G  224K  6.3G   1% /run/user/1000

As we can see in the output, the mount point / is an ext4 file system, with an NVMe SSD with a capacity of 1.8 TB, of which approximately 1.2 TB is still free. If other storage devices, such as external hard drives or USB drives, were present, they would also be included in the list. It certainly takes some practice to sharpen your eye for the relevant details. In the next step, we’ll practice a little diagnostics.

lsblk

If the output of df is too confusing, you can also use the lsblk tool, which provides a more understandable listing for beginners.

ed@local:~$ sudo lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
loop0          7:0    0     4K  1 loop /snap/bare/5
loop1          7:1    0  73.9M  1 loop /snap/core22/2139
loop2          7:2    0 516.2M  1 loop /snap/gnome-42-2204/226
loop3          7:3    0  91.7M  1 loop /snap/gtk-common-themes/1535
loop4          7:4    0  10.8M  1 loop /snap/snap-store/1270
loop5          7:5    0  50.9M  1 loop /snap/snapd/25577
loop6          7:6    0  73.9M  1 loop /snap/core22/2133
loop7          7:7    0  50.8M  1 loop /snap/snapd/25202
loop8          7:8    0   4.2G  0 loop 
└─veracrypt1 254:0    0   4.2G  0 dm   /media/veracrypt1
sda            8:0    1 119.1G  0 disk 
└─sda1         8:1    1 119.1G  0 part 
sr0           11:0    1  1024M  0 rom  
nvme0n1      259:0    0   1.9T  0 disk 
├─nvme0n1p1  259:1    0   976M  0 part /boot/efi
├─nvme0n1p2  259:2    0   1.8T  0 part /
└─nvme0n1p3  259:3    0  63.7G  0 part [SWAP]

S.M.A.R.T

To thoroughly test our newly acquired storage before use, we’ll use the so-called S.M.A.R.T (Self-Monitoring, Analysis and Reporting Technology) tools. This can be done either with the Disks program introduced in the first part of this article or with more detailed information via Bash. With df -hT, we’ve already identified the SSD /dev/nvme0, so we can call smartctl.

ed@local:~$ sudo smartctl --all /dev/nvme0 
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.48+deb13-amd64] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SAMSUNG MZVL22T0HDLB-00BLL
Serial Number:                      S75ZNE0W602153
Firmware Version:                   6L2QGXD7
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 2,048,408,248,320 [2.04 TB]
Unallocated NVM Capacity:           0
Controller ID:                      6
NVMe Version:                       1.3
Number of Namespaces:               1
Namespace 1 Size/Capacity:          2,048,408,248,320 [2.04 TB]
Namespace 1 Utilization:            248,372,908,032 [248 GB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            002538 b63101bf9d
Local Time is:                      Sat Oct 25 08:07:32 2025 CST
Firmware Updates (0x16):            3 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x0057):     Comp Wr_Unc DS_Mngmt Sav/Sel_Feat Timestmp
Log Page Attributes (0x0e):         Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg
Maximum Data Transfer Size:         128 Pages
Warning  Comp. Temp. Threshold:     83 Celsius
Critical Comp. Temp. Threshold:     85 Celsius

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     8.41W       -        -    0  0  0  0        0       0
 1 +     8.41W       -        -    1  1  1  1        0     200
 2 +     8.41W       -        -    2  2  2  2        0     200
 3 -   0.0500W       -        -    3  3  3  3     2000    1200
 4 -   0.0050W       -        -    4  4  4  4      500    9500

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        37 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    43,047,167 [22.0 TB]
Data Units Written:                 25,888,438 [13.2 TB]
Host Read Commands:                 314,004,907
Host Write Commands:                229,795,952
Controller Busy Time:               2,168
Power Cycles:                       1,331
Power On Hours:                     663
Unsafe Shutdowns:                   116
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               37 Celsius
Temperature Sensor 2:               37 Celsius

Error Information (NVMe Log 0x01, 16 of 64 entries)
No Errors Logged

Self-test Log (NVMe Log 0x06)
Self-test status: No self-test in progress
No Self-tests Logged

A very useful source of information, especially if you’re planning to install a used drive. Fortunately, my system SSD shows no abnormalities after almost two years of use.

fdisk

The classic among hard drive programs is fdisk, which is also available for Windows systems. With fdisk, you can not only format drives but also extract some information. For this purpose, there are the -l parameter for list and -x for more details. The fdisk program is quite complex, and for formatting disks, I recommend the graphical versions Disks and Gparted, presented in the first part of this article. With the graphical interface, the likelihood of making mistakes is much lower than with the shell.

ed@local:~$ sudo fdisk -l
Disk /dev/nvme0n1: 1.86 TiB, 2048408248320 bytes, 4000797360 sectors
Disk model: SAMSUNG MZVL22T0HDLB-00BLL              
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 347D3F20-0228-436D-9864-22A5D36039D9

Device              Start        End    Sectors  Size Type
/dev/nvme0n1p1       2048    2000895    1998848  976M EFI System
/dev/nvme0n1p2    2000896 3867305983 3865305088  1.8T Linux filesystem
/dev/nvme0n1p3 3867305984 4000796671  133490688 63.7G Linux swap

Disk /dev/sda: 119.08 GiB, 127865454592 bytes, 249737216 sectors
Disk model: Storage Device  
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xa82a04bd

Device     Boot Start       End   Sectors   Size Id Type
/dev/sda1        2048 249737215 249735168 119.1G 83 Linux

Basic repair with consistency check: fsck

If, contrary to expectations, problems arise, you can use the fsck (File System Consistency Check) tool to check the file system and repair it if necessary. However, you must specify the relevant partition.

sudo fsck /dev/sdc

As you can see in the screenshot, some time ago I had a partition with a defective superblock, making it impossible to access the data. The reason for the error was a defective memory cell in the SSD. With a little effort, I was able to access the data and copy it to another storage device. This doesn’t always work. Therefore, I would like to give a little advice: always be well prepared before such rescue operations. This means having a sufficiently large, functioning target drive ready so that you can immediately create a backup if the operation is successful. Many of the operations presented here change the data on the storage device, and it is not certain whether subsequent access will be successful.

Linux sysadmin joke

Linux system administrators often recommend that beginners delete the French language files, which aren’t needed, to save space. To do this, type the command sudo rm -fr / in the console and press Enter. This should not be done under any circumstances, as the command deletes the entire hard drive. It is considered the most dangerous thing you can do in Linux. You initiate the deletion with rm, the parameters -f and -r stand for force and recursive, respectively, and the inconspicuous / refers to the root directory.

Fake Check

Sometimes it happens that you’ve purchased storage devices that claim a high capacity, but that capacity isn’t even close to being there. These are so-called fake devices. The problem with these fake devices is that the data written to the device for which there is no longer any capacity ends up in oblivion. Unfortunately, you don’t receive an error message and often only notice the problem when you want to access the data again at a later time.

A very unpleasant way to obtain fake devices is through an Amazon hack. To ensure reliable and fast shipping, Amazon offers its sellers the option of storing goods in its own fulfillment center. Sellers who use this option are also given priority on the Amazon website. The problem is that the same products all end up in the same box, which makes perfect sense. Criminals shamelessly exploit this situation and send their fake products to Amazon. Afterwards, it’s impossible to identify the original supplier.

Abonnement / Subscription

[English] This content is only available to subscribers.

[Deutsch] Diese Inhalte sind nur für Abonnenten verfügbar.

Network Attached Storage (NAS)

Another scenario for dealing with mass storage under Linux involves so-called NAS hard drives, which are connected to the router via a network cable and are then available to all devices such as televisions, etc. To ensure that access to the files is only granted to authorized users, a password can be set. Affordable solutions for home use are available, for example, from Western Digital with its MyCloud product series. It would be very practical if you could automatically register your NAS during the boot process on your Linux system, so that it can be used immediately without further login. To do this, you need to determine the NAS URL, for example, from the user manual or via a network scan. Once you have all the necessary information such as the URL/IP, login name, and password, you can register the NAS with an entry in the /etc/fstab file. We already learned about the fstab file in the section on the SWAP file.

First, we install NFS support for Linux to ensure easy access to the file systems commonly used in NAS systems.

Abonnement / Subscription

[English] This content is only available to subscribers.

[Deutsch] Diese Inhalte sind nur für Abonnenten verfügbar.

In the next step, we need to create a file that enables automatic login. We’ll call this file nas-login and save it to /etc/nas-login. The contents of this file are our login information.

user=accountname
password=s3cr3t

Finally, we edit the fstab file and add the following information as the last line:

Abonnement / Subscription

[English] This content is only available to subscribers.

[Deutsch] Diese Inhalte sind nur für Abonnenten verfügbar.

The example is for a Western Digital MyCloud drive, accessible via the URL //wdmycloud.local/account. The account must be configured for the corresponding user. The mount point under Linux is /media/nas. In most cases, you must create this directory beforehand with sudo mkdir /media/nas. In the credentials, we enter the file with our login information /etc/nas-login. After a reboot, the NAS storage will be displayed in the file browser and can be used. Depending on the network speed, this can extend the boot process by a few seconds. It takes even longer if the computer is not connected to the home network and the NAS is unavailable. You can also build your own NAS with a Raspberry Pi, but that could be the subject of a future article.

Finally, I would like to say a few words about Western Digital’s service. I have already had two replacement devices, which were replaced by WD every time without any problems. Of course, I sent the service department my analysis in advance via screenshot, which ruled out any improper use on my part. The techniques presented in this article have helped me a great deal, which is why I came up with the idea of ​​writing this text in the first place. I’ll leave it at that and hope the information gathered here is as helpful to you as it was to me.


Take professional screenshots

Take professional screenshots

Neo Diseno Sep 29, 2025 5 min read

Sure, you can take a screenshot with your smartphone, but creating screenshots isn’t always that easy. In this short article,…

Pathfinder

So that we can call console programs directly across the system without having to specify the full path, we use the so-called path variable. So we save the entire path including the executable program, the so-called executable, in this path variable so that we no longer have to specify the path including the executable on the command line. By the way, the word executable derives the file extension exe, which is common in Windows. Here we also have a significant difference between the two operating systems Windows and Linux. While Windows knows whether it is a pure ASCII text file or an executable file via the file extension such as exe or txt, Linux uses the file’s meta information to make this distinction. That’s why it’s rather unusual to use these file extensions txt and exe under Linux.

Typical use cases for setting the path variable are programming languages ​​such as Java or tools such as the Maven build tool. For example, if we downloaded Maven from the official homepage, we can unpack the program anywhere on our system. On Linux the location could be /opt/maven and on Microsoft Windows it could be C:/Program Files/Maven. In this installation directory there is a subdirectory /bin in which the executable programs are located. The executable for Maven is called mvn and in order to output the version, under Linux without the entry in the path variable the command would be as follows: /opt/maven/bin/mvn -v. So it’s a bit long, as we can certainly admit. Entering the Maven installation directory in the path shortens the entire command to mvn -v. By the way, this mechanism applies to all programs that we use as a command in the console.

Before I get to how the path variable can be adjusted under Linux and Windows, I would like to introduce another concept, the system variable. System variables are global variables that are available to us in Bash. The path variable also counts as a system variable. Another system variable is HOME, which points to the logged in user’s home directory. System variables are capitalized and words are separated with an underscore. For our example with entering the Maven Executable in the path, we can also set our own system variable. The M2_HOME convention applies to Maven and JAVA_HOME applies to Java. As a best practice, you bind the installation directory to a system variable and then use the self-defined system variable to expand the path. This approach is quite typical for system administrators who simplify their server installation using system variables. Because these system variables are global and can also be read by automation scripts.

The command line, also known as shell, bash, console and terminal, offers an easy way to output the value of the system variable with echo. Using the example of the path variable, we can immediately see the difference to Linux and Windows. Linux: echo $PATH Windows: echo %PATH%

ed@local:~$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/snap/bin:/home/ed/Programs/maven/bin:/home/ed/.local/share/gem//bin:/home/ed/.local/bin:/usr/share/openjfx/lib

Let’s start with the simplest way to set the path variable. In Linux we just need to edit the hidden .bashrc file. At the end of the file we add the following lines and save the content.

export M2_HOME="/opt/maven"
export PATH=$PATH:$M2_HOME/bin

We bind the installation directory to the M2_HOME variable. We then expand the path variable to include the M2_HOME system variable with the addition of the subdirectory of executable files. This procedure is also common on Windows systems, as it allows the installation path of an application to be found and adjusted more quickly. After modifying the .bashrc file, the terminal must be restarted for the changes to take effect. This procedure ensures that the entries are not lost even after the computer is restarted.

Under Windows, the challenge is simply to find the input mask where the system variables can be set. In this article I will limit myself to the version for Windows 11. It may of course be that the way to edit the system variables has changed in a future update. There are slight variations between the individual Windows versions. The setting then applies to both the CMD and PowerShell. The screenshot below shows how to access the system settings in Windows 11.

To do this, we right-click on an empty area on the desktop and select the System entry. In the System – About submenu you will find the system settings, which open the System properties popup. In the system settings we press the Environment Variables button to get the final input mask. After making the appropriate adjustments, the console must also be restarted for the changes to take effect.

In this little help, we learned about the purpose of system variables and how to store them permanently on Linux and Windows. We can then quickly check the success of our efforts in the shell using echo by outputting the contents of the variables. And we are now one step closer to becoming an IT professional.


Process documents with AnythingLLM and Ollama

We already have a guide with GPT4all on how to run your own local LLM. Unfortunately, the previous solution has a small limitation. It cannot process documents such as PDFs. In this new workshop, we will install AnythingLLM with Ollama to be able to analyze documents.

The minimum requirement for this workshop is a computer with 16 GB of RAM, ideally with Linux (Mint, Ubuntu, or Debian) installed. With a few adjustments, this guide can also be followed on Windows and Apple computers. The lower the hardware resources, the longer the response times.

Let’s start with the first step and install Ollama. To do this, open Bash and use the following command: sudo curl -fsSL https://ollama.com/install.sh | sh. This command downloads Ollama and executes the installation script. For the installation to begin, you must enter the administrator password. Ollama is a command-line program that is controlled via the console. After successful installation, a language model must be loaded. Corresponding models can be found on the website https://ollama.com/search.

Proven language models include:

  • lama 3.1 8B: Powerful for more demanding applications.
  • Phi-3-5 3B: Well-suited for logical reasoning and multilingualism.
  • Llama 3.3 2B: Efficient for applications with limited resources.
  • Phi 4 14B: State-of-the-art model with increased hardware requirements but performance comparable to significantly larger models.

Once you’ve chosen a language model, you can copy the corresponding command from the overview and enter it into the terminal. For our example, this will be DeepSeek R1 for demonstration purposes.

As shown in the screenshot, the corresponding command we need to install the model locally in Ollama is: ollama run deepseek-r1. Installing the language model may take some time, depending on your internet connection and computer speed. Once the model has been installed locally in Ollama, we can close the terminal and move on to the next step: installing AnythingLLM.

Installing AnythingLLm is similar to installing Ollama. To do so, open the terminal and enter the following command: curl -fsSL https://cdn.anythingllm.com/latest/installer.sh | sh. Once the installation is complete, you can change to the installation directory, which is usually /home//AnythingLLMDesktop. There, navigate to the start link and make it executable (right-click and select Properties). Additionally, you can create a shortcut on the desktop. Now you can conveniently launch AnythingLLM from the desktop, which we’ll do right now.

After defining the workspace, we can now link Anything with Ollama. To do this, we go to the small wrench icon (Settings) in the lower left corner. There, we select LLM and then Ollama. We can now select the language model stored for Ollama. Save our settings. Now you can switch to chat mode. Of course, you can change the language model at any time. Unlike previous workshops, we can now upload PDF documents and ask questions about the content. Have fun.


Treasure chest – Part 2

In the previous part of the article treasure chest, I described how the database connection for the TP-CORE library got established. Also I gave a insight to the internal structure of the ConfiguartionDO. Now in the second part I explain the ConfiguartionDAO and its corresponding service. With all this knowledge you able to include the application configuration feature of TP-CORE in your own project to build your own configuration registry.

Lets resume in short the architectural design of the TP-CORE library and where the fragments of the features located. TP-CORE is organized as layer architecture as shown in the graphic below.

As you can see there are three relevant packages (layer) we have to pay attention. As first the business layer resides like all other layers in an equal named package. The whole API of TP-CORE is defined by interfaces and stored in the business layer. The implementation of the defined interfaces are placed in the application layer. Domain Objects are simple data classes and placed in the domain layer. Another important pattern is heavily used in the TP-CORE library is the Data Access Object (DAO).

The GenericDAO provides the basic CRUD operations, we don’t need to repeat in every specialization again. A full description how the GenericDAO is implemented I documented in the GitHub Wiki of TP-CORE.

Now the days micro services and RESTful application are state of the art. Especially in TP-CORE the defined services aren’t REST. This design decision is based on the mind that TP-CORE is a dependency and not a standalone service. Maybe in future, after I got more feedback how and where this library is used, I could rethink the current concept. For now we treat TP-CORE as what it is, a library. That implies for the usage in your project, you can replace, overwrite, extend or wrap the basic implementation of the ConfigurationDAO to your special necessities.

To keep the portability of changing the DBMS Hibernate (HBM) is used as JPA implementation and O/R mapper. The Spring configuration for Hibernate uses the EntityManager instead of the Session, to send requests to the DBMS. Since version 5 Hibernate use the JPA 2 standard to formulate queries.

As I already mentioned, the application configuration feature of TP-CORE is implemented as DAO. The domain object and the database connection was topic of the first part of this article. Now I discuss how to give access to the domain object with the ConfigurationDAO and its implementation ConfigurationHbmDAO. The domain object ConfigurationDO or a list of domain objects will be in general the return value of the DAO. Actions like create are void and throw just an exception in the case of a failure. For a better style the return type is defined as Boolean. This simplifies also writing unit tests.

Sometimes it could be necessary to overwrite a basic implementation. A common scenario is a protected delete. For example: a requirement exist that a special entry is protected against a unwanted deletion. The most easy solution is to overwrite the delete whit a statement, refuses every time a request to delete a domain object whit a specific UUID. Only adding a new method like protectedDelete() is not a god idea, because a developer could use by accident the default delete method and the protected objects are not protected anymore. To avoid this problem you should prefer the possibility of overwriting GenericDAO methods.

As default query to fetch an object, the identifier defined as primary key (PK) is used. A simple expression fetching an object is written in the find method of the GenericHbmDAO. In the specialization as ConfigurationHbmDAO are more complex queries formulated. To keep a good design it is important to avoid any native SQL. Listing 1 shows fetch operations.

//GenericHbmDAO
public T find(final PK id) {
  return mainEntityManagerFactory.find(genericType, id);
}
//ConfigurationHbmDAO
public List getAllConfigurationSetEntries(final String module,
final String version, final String configSet) {

  CriteriaBuilder builder = mainEntityManagerFactory.getCriteriaBuilder();
  CriteriaQuery query = builder.createQuery(ConfigurationDO.class);
  // create Criteria
  Root root = query.from(ConfigurationDO.class);
  query.where(
    builder.equal(root.get("modulName"), module),
    builder.equal(root.get("version"), version),
    builder.equal(root.get("configurationSet"), configSet)
  );
  return mainEntityManagerFactory.createQuery(query).getResultList();
}
Java

The readability of these few lines of source is pretty easy. The query we formulated for getAllConfigurationSetEntries() returns a list of ConfigurationDO objects from the same module whit equal version of a configSet. A module is for example the library TP-CORE it self or an ACL and so on. The configSet is a namespace that describes configuration entries they belong together like a bundle and will used in a service like e-mail. The version is related to the service. If in future some changes needed the version number have increase. Lets get a bit closer to see how the e-mail example will work in particular.

We assume that a e-mail service in the module TP-CORE contains the configuration entries: mailer.host, mailer.port, user and password. As first we define the module=core, configSet=email and version=1. If we call now getAllConfigurationSetEntries(core, 1, email); the result is a list of four domain objects with the entries for mailer.host, mailer.port, user and password. If in a newer version of the email service more configuration entries will needed, a new version will defined. It is very important that in the database the already exiting entries for the mail service will be duplicated with the new version number. Of course as effect the registry table will grow continual, but with a stable and well planned development process those changes occur not that often. The TP-CORE library contains an simple SMTP Mailer which is using the ConfigurationDAO. If you wish to investigate the usage by the MailClient real world example you can have a look on the official documentation in the TP-CORE GitHub Wiki.

The benefit of duplicate all existing entries of a service, when the service configuration got changed is that a history is created. In the case of update a whole application it is now possible to compare the entries of a service by version to decide exist changes they take effect to the application. In practical usage this feature is very helpful, but it will not avoid that updates could change our actual configuration by accident. To solve this problem the domain object has two different entries for the configuration value: default and configuration.

The application configuration follows the convention over configuration paradigm. Each service need by definition for all existing configuration entries a fix defined default value. Those default values can’t changed itself but when the value in the ConfigurationDO is set then the defaultValue entry will ignored. If an application have to be updated its also necessary to support a procedure to capture all custom changes of the updated configuration set and restore them in the new service version. The basic functionality (API) for application configuration in TP-CORE release 3.0 is:

  • void updateConfigurationEntries(List<ConfigurationDO> configuration)
  • ConfigurationDO getConfigurationByKey( String key, String module, String version)
  • List<ConfigurationDO> getAllConfigurationSetEntries(String module, String version, String configSet)
  • List<ConfigurationDO> getAllModuleEntries(String module)
  • List<ConfigurationDO> getAllDeprecatedEntries()
  • List<ConfigurationDO> getHistoryOfAEntry(String module, String key, String configSet)
  • String getValueByKey(String key, String module, String version)
  • void restoreKeyToDefault(ConfigurationDO entry)

The following listing gives you an idea how a implementation in your own service could look like. This snipped is taken from the JavaMailClient and shows how the internal processing of the fetched ConfigurationDO objects are managed.

private void processConfiguration() {
    List configurationEntries =  
        configurationDAO.getAllConfigurationSetEntries("core", 1, "email");

    for (ConfigurationDO entry : configurationEntries) {
        String value;
        if (StringUtils.isEmpty(entry.getValue())) {
            value = <strong>entry.getDefaultValue</strong>();
        } else {
            value = <strong>entry.getValue</strong>();
        }

        if (entry.getKey()
                .equals(cryptoTools.calculateHash("mailer.host",
                        HashAlgorithm.SHA256))) {
            configuration.replace("mailer.host", value);
        } else if (entry.getKey()
                .equals(cryptoTools.calculateHash("mailer.port",
                        HashAlgorithm.SHA256))) {
            configuration.replace("mailer.port", value);
        } else if (entry.getKey()
                .equals(cryptoTools.calculateHash("user",
                        HashAlgorithm.SHA256))) {
            configuration.replace("mailer.user", value);
        } else if (entry.getKey()
                .equals(cryptoTools.calculateHash("password",
                        HashAlgorithm.SHA256))) {
            configuration.replace("mailer.password", value);
        }
    }
}
Java

Another functionality of the application configuration is located in the service layer. The ConfigurationService operates on the module perspective. The current methods resetModuleToDefault() and filterMandatoryFieldsOfConfigSet() already give a good impression what that means.

If you take a look on the MailClientService you detect the method updateDatabaseConfiguration(). May you wonder why this method is not part of the ConfigurationService? Of course this intention in general is not wrong, but in this specific implementation is the update functionality specialized to the MailClient configuration. The basic idea of the configuration layer is to combine several DAO objects to a composed functionality. The orchestration layer is the correct place to combine services together as a complex process.

Resume

The implementation of the application configuration inside the small library TP-CORE allows to define an application wide configuration registry. This works also in the case the application has a distribute architecture like micro services. The usage is quite simple and can easily extended to own needs. The proof that the idea is well working shows the real world usage in the MailClient and FeatureToggle implementation of TP-CORE.

I hope this article was helpful and may you also like to use TP-CORE in your own project. Feel free to do that, because of the Apache 2 license is also no restriction for commercial usage. If you have some suggestions feel free to leave a comment or give a thumbs up.