Literate Programming in GO

The GO Programming

GO, started in 2008 by Google, is a project designed while waiting for the C++ program to compile. It is a half-truth based on research sources. It got conceived and designed as a solution to the dissatisfaction with the development environments used at Google. GO comes with an engaging object model as it is an Object-Oriented-Model. The lynchpin of this program is the interfaces involved. It is a flexible model which is easy to operate. Now, GO is used for Web Programming as it has become critical to the success of many businesses.

Literate Programming: A Team Sport!

Any program is a story written by integrating two vital components referred to as the Code and Prose. GitHub is a canvas that contains countless programs which get updated every day. In the mid-1980s, research projects went on in the field of architectural document software were conducted by the Army. As part of the Milspec Software Documentation, certain architectural documents got produced wherein the output got dwarfed by piles of boilerplate extracted from the Source Code. The research took an interesting turn, when the data amounted to literally nothing significant, which screamed the need for a direction change.

As the researchers were keen observants, they started following the scientific adventures of Donald Knuth, who developed an idea known as Literate Programming. David Knuth quoted, “A program is a story told in two languages: Code and Prose. The developer is required to write the Code and Prose at the same time in a way that, the Prose gets elevated to the status of the Code. To improve upon this research ideas got explored in the betterment of Prose narration at varying levels of abstraction that were gradually fleshed out, in the Code. For programs to work this way, a tool got developed that integrated the two rather diligently. TeX, the language in which the Prose got expressed could be typeset with proper scientific beauty. Pascal, another language expressed Code. These both got compiled into a Literate Program. Here one tool extracted Prose whereas; the other took care of Code for proper compilation. This idea got implemented over the years for a few different pairs of other programming languages as well.

Although this idea by Donald Knuth clicked only in small circles earlier, lately it has taken off big time creating a great impetus in the field of Literate Programming. Modifications have occurred as part of the evolution of this idea as clearly seen on GitHub where countless stories of programs get told every day. There are some ways by which non-programmers get drawn into the culture of GitHub. The literary culture draws inspiration from inline instruments that are a form of expression through which stories of codes get told in commit messages and issue discussions.

Clean Commit History fascinates scholars from various other disciplines as well. In the current era of always-on and continuously improving software services designed and developed by an evolving team of developers & contributors, the data is as clear as the day with no secrets, tricks, or mystery elements. Literate Programming has become more fluid and flexible wherein anyone can fix a bug, add a feature, or integrate with other services while understanding the existing elements and the way they became crucial.

The programming history gets summarized in various streams of commit messages. So, it becomes so important to have huge databases on how to write it with due diligence. Some of the questions that arise include whether should you merge the changes to preserve the graph of branches that precedes a commit or if a developer’s personal style is to commit more frequent & granular changes that help out a future collaborator? Also, it should be thought about how the developer rebases to flatten a graph of branches that precedes a commit? Ultimately, is it necessary to aggregate all the changes into bigger chunks for the ease of pushing into its evolutionary cycle? These great questions form the perfect starting point and crux for somebody working in digital humanities to explore. This literary programming form is inherently collaborative since the 1980s when Donald Knuth first envisioned Literate Programming and devised its ideologies & principles. It was not unusual for a major software project such as TeX to emerge from the mind of a brilliant scientist like David Knuth. The technological era brought in evolved times wherein Software Development is much more deeply understood and more likely a Team Sport rather than an individual prowess.

Literate Programming in GO

Here, a “code-as-wiki” approach was undertaken which falls short of the standard set by the brilliant scientist, Donald Knuth. However, this step leads us in the right direction while performing Literate Programming in GO.

Usually, company hackathons serve as a great place to analyze the various nuances of programming. A conscious effort was put in to improve the documentation for the Steampipe Plugin SDK. Like all the other components in the Steampipe System, the SDK Plugin is written in GO and published to pkg.go.dev. The required documentation gets auto-generated through a catalog of functions and types. The hackathon challenge devised by this particular company or organization served to weave proper guidance into the generated documentation. For instance, the list of tasks considered for writing a Steampipe Plugin looks like defining the plugin, creating the plugin entry point, and then defining the first table before subsequent tasks. All these tasks rely on various functions and types. The comments attached to all these functions and types, enhance the generated documentation. These are too granular for high-level exposition that gets aimed for during hackathons. In this hackathon, the company provided inspiration through a top-level page that lacked the overview section in the documentation for pgx, a GO Programming driver for Postgres.

Writing the wiki in GO Comments

For this purpose, a file name doc.go got used as it is conventional; although, it could have been called foo.go. The salient feature of the file name is that it is a valid GO file which belongs to a package. This package contains documentation and no code. Headers were written to create the sections of overview. Each of these sections contains a plugin that includes the writer’s task using proper narrative, inline code examples, internal links to functions and types, and external links to other related & relevant examples. Here, the ability to link within a code’s namespace was indeed a revelation. The ability to create new wiki pages was not fulfilled and this created hindrances in explaining higher-level concepts. The overview is a place to do so but work needs to be done to reserve that for the narration of the plugin writer’s journey.

A place is needed to discuss more about the concept of dynamic tables. An advanced feature gets introduced that enables plugins like CSV that have no fixed schema and must define columns on the fly. Only expert companies could create the new documentation-only packages for all tricky topics. The developers can import the data so that all names become available for the same kind of shorthand linking. The authorizing experience fully feels more wiki-like. Creating new pages has become easier as needed while weaving them into the hypertextual documentation that lives within the code base. Admittedly, it was indeed a struggle to use the GO Programming system. Here, the common syntax is Markdown-like but not Markdown as it depends on many implicit formatting conventions. For proceeding through this route, the programmers and developers need a local previewing tool. However, it is not super obvious that the one tool a programmer or developer wants is go.doc but rather pkgsite.

How is this literate programming?

According to David Knuth, the practitioner of literate programming gets regarded as an essayist. The main concern here is with exposition and excellence of style. To support his article and the concept of literate programming, David Knuth invented a system known as web with two components named tangle and weave. This enabled the author of a program or to tell its story in a language of mixed code and documentation in a narrative way in the first-person tone. The modern ways of mixing code & documentation helps with generating documents from embedded comments that superficially resemble David Knuth’s practice. The hackathon exercise served the purpose of observing what code-s-wiki can become. However, it does feel like a step toward the storytelling approach that David Knuth advocates. The genius researcher named the system as web before the actual internet got developed. Ultimately, this way of improving the narrative quality of auto-generated documents does feel like a step in the right direction.

Conclusion

GO language and literate programming go hand-in-hand as per many pioneers in the field. It is wonderful to see how GO developed by Google integrates well with literate programming.