The Orchid Manor

Wherein ideas bloom, knowledge flourishes, and the unknown dissipates.

Holy Grail Projects

While talking to my younger brother, he said off-handedly that he knew I'd been thinking deeply about an idea, that for me, would be my life's work, but he referred to it as my Holy Grail Project, and that phrase so accurately captures how much the project means to me, that it has become a meta-North Star, if you will.

The project (which I'll give a brief overview of later), is a North Star, something that guides and orients me in life- however I believe one can have many North Stars: what makes a North Star a Holy Grail Project is that it is something that the best version of you would be willing to suffer for, and pour your very living essence into.

We all have things that we're driven towards, and drawn to, over most others, but I would ask you to oblige me in a small thought experiment, and consider what of those things for you, would the best version of you be willing to suffer for, and pour ones lifeblood into?

If nothing in that list of things meets this criteria, that's fine, as they can still be productive, enriching, nourishing to the soul, worthy of time, effort and resource expenditure, however perhaps reflect on the degree to which they really are, to you, and reclibrate the values accordingly.

As well, take some time to contemplate, soul search, and meditate, on what a project that meets this criteria for you would be? Are there any ideas that come to mind? It could be that the endeavor that satisfies these criteria intimidates you, which is perfectly fine, in some ways, it should to some degree.

While I don't think everyone must have a Holy Grail Project, I think, if you can identify one, that you shouldn't shy away from it, because there's something within you that wants to achieve something outsized, and more importantly wants you to be the best version of you, while doing so.

There's nothing wrong with having multiple Holy Grail Projects either, Elon has at least three: multi-planetary civilization, AI, and freedom of speech/anti-censorship.

So, let's do a quick recap of what constitutes a Holy Grail Project.

It requires you to become the best version of you.
The best version of you must be willing to suffer to complete it.
It demands that you pour the essence of yourself into it, and you must find it worthy of having that demand met.

I'd like to add one more condition, or property rather, which I think is the most important:

It must outlive you.

The reasoning behind this last condition is, if the best version of you is going to sacrifice a meaningful amount of the limited time you have here, and your very essence, it should, much like the actual Holy Grail, be something that people value, seek out, and desire, long after you're gone.

There's no way to know if that will be so, unless your successful Holy Grail Project is conquering death itself, in which case this condition no longer matters because no one will die, or at least, no one will remain dead. Though on the other hand, it still counts because that would certainly be something that would be valued long after what was once the normative lifespan.

Also, it doesn't have to be at the scale of making civilization space-faring, it could be something commonplace, such as raising children, which, while commonplace, is nonetheless a priceless, noble, and inestimably meaningful endeavor to undertake.

Below, you'll find a short overview of my Holy Grail Project, ArtRiver.

Purpose

The Dilemma of Audio: Many Versions, No Control

Version control is not at all a new concept for iterative endeavors, having been used in industrial manufacturing and design, before it was ported to the domain of programming.

However, given the meteoric rise of complex software, it has become genericized, with the star player being git. In less than a century from the genesis of the field, programming has effectively solved the issue of version control.

Now, it is time for the creative domain to evolve, and adopt version control, as a integral part of its existence. A quick look around the internet will make it clear that the creative domain, has not progressed beyond First-generation version control, simply saving file names with appended and incremented series of numbers, and sharing work by emailing files.

While that may work for individual local projects, it's inefficient, unwieldy, and makes collaboration painful, providing no way to way to synchronize changes, and combine work done on various parts of creative projects. It offers no utility in the way back-ups, leading to a common problem in, for example, the music industry, where files are lost forever.

In 2008, a fire at Universal Studios Hollywood destroyed roughly one to two hundred thousand master recordings, belonging to iconic artists and bands like Elton John, Nirvana, Janet Jackson, Niel Diamond, Aerosmith, Aretha Franklin, Ray Charles, Lynyrd Skynyrd and Slayer.

Lucian Grainge, the CEO of Universal Music Group said in response to the event that "...the loss of even a single piece of archived material is heartbreaking."

He's correct.

ArtRiver exists to make it so that none of these problems afflict the creative process any longer. No one, from the humble hobbyists, to the greatest artists of our time, and times gone by, should lose entire pieces of artwork, due to freak accidents, and human error.

In an era where the creation of art is highly technological, and iconic art from the analog era becomes digitized, version control, which is so pivotal in creating the tools for making and translating art to a new medium, has a powerful and necessary role to play.

Overview

May We All Meander More

ArtRiver is a distributed version control system for musicians, that will allow artists to track and navigate the various branches and possibilities a creative work went through, on the way to completion.

Artriver exists to allow artists to draw from a granular record of the various tweaks and changes they make to various pieces of art, by coalescing these creative excursions in various directions into a navigable digital construct.

Artists can avoid the worry of accidental deletions, not just of whole works, but of parts of versions of works, and the like. From another perspective, it offers a new methodology to hone their craft, pin-pointing the mixing or synthesizer settings that didn't resonate, blossom fully, or highlight those that put the art on the path to becoming a finished product.

The creative process, rather than culminating in this point, can in fact have a life beyond it.

Concepts

What Makes ArtRiver Run

The ArtRiver.

Every creative project is an ArtRiver, which will exist locally.

The Timeshore.

For each ArtRiver project, there is an associated Timeshore, a metadata focused data structure that will chronicle major and minor changes, and save points, throughout the lifetime of an ArtRiver.

The Glacier.

Once a change to an ArtRiver project is finalized, it will be preserved in the Glacier, a data construct which mirrors an ArtRiver, and applies changes from the Timeshore. It is analagous to a remote, and allows for collaboration and synchronization/distribution of changes to ArtRiver project.

The Iceberg.

Once a change has been preserved in a Glacier, it will be stored in the Iceberg, a compressed replica of the project and it's history. It is a last resort redunancy solution, for extreme situations where remotes become corrupted or destroyed, and provides a preservation mechanism for master recordings of the great works of media, as they become digitized, remastered, etc.

Cratemap

What Lies Downstream

Plugin Engine: Flexible and robust functionality for hooking to the DAW, scanning for plugins of various types, and recording settings and preferences.

Metadata Engine: Metadata utilities for files, projects, and collaborators to bolster versioning functionality, access permissions, and state synchronization.

Access Control Engine: Provision permissions for internal and external collaborators, according to role and area of expertise, or projects, and the files that comprise them.

Automation Engine: Tooling that will allow automating mixing and editing preferences, integrating them into into the workflow of project management, and applying effects on the fly.

ArtRiver's Raison D'etre

There are a couple of reasons why I think this should exist, as a bespoke solution, as opposed to just using Git to achieve the same result:

1. Git is deeply coupled to text, and programs: text is incredibly different from audio, and programs are incredibly different from projects, and DAWs are incredibly different from IDEs/text editors.

2. Git has a file size limitation that can easily be blown past by meaningful audio projects, and while there is GitLFS, it is limited to 2GB, and if you want to remove LFS objects, you have to delete and recreate the entire repository.

3. Git can track project files, but getting git to track various pieces of an audio project, such as synthesizer and mixing board settings, could be incredibly messy, and difficult, as opposed to a DVCS designed to be aware of such things.

3.1. Git is not designed for interfacing with audio hardware: if I'm using a daw, but have physical synthesizers, such as a minimoog, that adds an additional layer of messiness, as mentioned above.

4. In the event that I use git to track a project done in Audacity, I have an .aup3 file, which is recognizeable by Audacity, bit I cannot then reopen this project and access it's history in Ableton: This would require an abstraction layer that would have to be bolted onto Git. A bespoke solution would allow for the abstraction layer to handle conversions between DAWs, and their project file structure, setup, and encoding.

5. Getting musicians to learn Git seems an order of magnitude more difficult than than a DVCS that is crafted specifically for their tools as they already exist.

6. Git uses snapshot storage, and I prefer delta-storage.

7. Using Git naturally requires musicians to use Github, to update collaborators versions of projects, and, while it is amazing for code, it simply does not appeal to the musician in me, and it doesn't do much in the way of allowing for visualization: I can view the diff between two versions of the same text file, but there's no facility for diffing audio tracks and seeing that the dB level, EQ settings, etc, were changed. Personally, as a musician, I would like something that at least slightly resembles a DAW.

8. Git syntax. I know, I know, skill issue, but if a solution that provides a simpler set of commands (for those inclined to use a CLI version of ArtRiver) could be created, why not? Many musicians are technical, and yes, some are audio programmers, but many are not. Should I release an ArtRiver plugin, it makes more sense to wrap commands built into it, than to wrap git commands and then present an in-DAW GUI that abstracts Git commands away, to create the illusion of them being made bespoke for audio project management.

9. Text/programs are naturally more discrete than audio projects, and audio itself, and changes in Git must be added and committed manually. I'd like a solution that tracks things in a more automated manner, so that cataloguing changes doesn't distrupt the creative process, which is continuous, and not amenable to logging discrete changes. Mixing and mastering engineers often take notes of settings but they're not the artist (which is who this is for primarily), and note taking is not version control, it's data collection.

There are more reasons I could list perhaps, but I think I've made my overall point: while Git certainly could be used to version audio projects, I believe a much more robust, flexible, powerful and enjoyable version control experience for audio projects requires a bespoke solution that must, from day one, be designed with audio in mind. A final tidbit is that, some DAWs, like Ableton, do provide pseudo-version control, but this is entirely within Ableton, and suffers from a vendor lock in of sorts, because I can't open that project in Logic and have access to my history. A solution that provides an abstraction/conversion layer, and exists outside of a given DAW, allows for greater freedom, and offers an artist greater control over their project.

Is this idea feasible? It is reasonable? Maybe, maybe not, and it will certainly change extensively as it becomes implemented and used, and mature: who knows what of the idea as expressed here will survive to become a part of the ArtRiver (pre)alpha release, and who knows what will be added.

At the end of the day, I'm a musician and a programmer, with a wild dream, and enough ambition to try it, and if it fails, at least I will learn volumes upon volumes about digital audio and programming. That alone makes it worth it to me.