everything wrong with free software
"obedience breeds foolishness"
=> the-wonders-of-modularity.html the-wonders-of-modularity
*originally posted:* apr 2021
this proposal may have a few elements of a design document, but should really be thought of as an open letter.
as to attribution, gnew deserves all of the credit and none of the blame. one of the goals of what the chief gnewsance and i are doing (for ourselves, at the very least) is to replace email. the goals of fmail at least, are much more humble than the goals of email.
if you think any of the following ideas are ridiculous, you should blame me and not gnew-- im throwing a lot of things in that only i should be blamed for, but i cant take all of the credit for some of the following either. talking with the chief gnewsance has helped to simplify some of this, and i wouldnt have likely considered these things if not for other ideas that come from gnew-- which are the subject of an upcoming paper. so when i say gnew should get all of the credit and none of the blame, thats really the best way to be fair about this.
with that disclaimer out of the way, i will describe some of the inspiration i take from gnew.
part of it is the idea of simplifying protocols. another aspect is to reduce reliance on projects that have grown overly complicated and co-opted. i dont have much against email as a technology, per se. if it were that bad, i would just stop using it-- i wouldnt try to reimplement or reuse any part of it in simpler terms unless it were partly useful to begin with.
the biggest problem with email is probably that it has been taken over. the protocol is ancient enough (more than is needed for fmail) but the implementation of email as a service is taken over by large companies in the name of fighting spam. while fighting spam is important, the way this is implemented (as a service) leads to large companies ultimately controlling who can email whom. for example, its proven effectively impossible for me to get an automated email from muckrights, even if i want one. i am no longer associated with them, however not that long ago this impossibility was an excellent example of how email is broken.
and its less that i want to "fix email"-- i just dont want to use something that is so broken, if an alternative is within practical reach. so the goal for me personally isnt to fix email, just to use some alternative.
with the muckrights example, it doesnt really matter if i couldnt get email from the website because they deserved to be flagged as spammers, or whether they were attacked by corporations in an effort to censor them. what matters to me most about it is that when i wanted the website itself to send me email, it couldnt. but there are other problems with email, many of which are related to centralisation. this centralisation is bolted on to a protocol which was designed specifically to be decentralised. lawrence lessig talks about such recentralisation in code 2.0-- and people who love the promises of freedom the internet used to represent tend to be against such recentralisation.
it may seem funny that fmail (the f stands for "file" but informally, you can call it "foldermail" if that sounds less awkward to you) is based on the concept of a local filesystem. that will hopefully make more sense after the next gnew paper is published, but what it means is that it doesnt matter if fmail is implemented via two people taking turns using the same physical computer, or whether one or both people are logged in remotely to the same filesystem, or whether the "local" filesystem itself is actually remote shares over ssh or similar. what matters to fmail is the abstraction of the filesystem. you could implement a system just like this quite easily in dos, if you wanted. this targets bsd, but is equally possible on gnu/linux (at least until systemd finds a way to break this, too).
a fairly reasonable question to ask is-- why not just use local email? we certainly could. i happen to hate the most common and readily available tools for doing that, and they are not designed to do exactly what i want to be able to do.
another question some clever person might ask would be-- why not just use git for this? someone may figure out, while reading this proposal, that git would be well-suited to this task. i almost agree with that.
like email itself, git would probably be overkill for this purpose. im not comfortable enough using git to replace email with it, and the processes outlined here are far more modest and ultimately require comfort with the local system-- not something featureful like git. but as to whether people who love git could use it for this purpose and skip fmail altogether? absolutely.
fmail is designed to be simple and have very few features, with a focus on (mostly) 1-to-1 communication-- and i basically never use cc so thats not even covered here. if you want cc maybe you can think of a way (or possibly just use local email) though git is a platform designed around versioning, project management and collaboration between many people at once. its certainly possible to simulate cc by copying your email to more than one folder. or, you could extend fmail to do this automatically, based on some text or option somewhere.
the basic idea is to have a folder (probably under your home folder) called "mail/"-- i prefer mail/ personally, though mbox/ was already used and fmail/ is perhaps most appropriate. in the mail/ folder are two folders ("directories" for purists, but i often favour shorter terms like "folder" and "coder" over their more traditional counterparts) named in/ and out/ which each contain a folder with a "username" for the system.
the username, as with email, is actually based on the login of the unix system user. for dos, hypothetically and rhetorically speaking, this could be whatever "username" is chosen as dos does not have user account logins.
thus alice1, if she only emailed bob3, would have folders like this:
you could design the whole thing as an inversion of this, like so:
however i dont prefer this, because i think it is less clumsy and more likely useful to traverse all users from in/ than to traverse all users but only the in/ subfolder of each. either would technically work, but with the first one you can simply find ~/mail/in/ to check email.
you may be thinking-- what about other folders like read/ and sent/? it is not necessary to specify read/ because either, the user creates a read folder, or does whatever they please (maybe they prefer to move read emails to a totally different place, like a usb drive) or they use a client that has its own idea of what to do with emails after they are read. we simply dont need to spec that at all!
very simple specifications dont have to spec what they dont need to do, and its not our concern what users (operators) do with read emails. that can be left up to the person using the shell, or using a client.
sent/ on the other hand, is a good place to put emails after the system (or user) has sent them. and perhaps we can imagine different clients handling that differently too, but the difference is that fmail is conceptualised as having one (or more) "reader" clients per user, but a single daemon (or at least a set of manually run scripts that expect this basic folder structure) per system.
thus it doesnt matter what read/ is called, but the system will probably either move everyones files from ./out/username to ./out/username/sent (this is my preference) or it will (a different design) rename every sent file to filename.sent in the same folder (this is my second choice) or it will encounter overhead by keeping a list of sent hashes and refusing to send anything with the same hash twice.
with the latter option, you have an interesting problem where if bob asks alice to send him a script, but he accidentally deletes it, the system will refuse to send bob the same script twice when he asks alice to send it again. she can fix this by padding the file, or we can add more overhead with date checks, but with the first two designs all she has to do is move the file from sent/ or rename it to drop .sent. i think the sent/ folder design is the simplest.
lets look at my preferred design in action:
after the root or fmail-privileged daemon (or manual scripts) traversed the out/ subfolder for each user, it would copy e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 to /home/bob3/mail/in/alice1 and move it to sent/ with the following results:
now when alice runs find on ~/mail/out/ with grep -v "\/sent\/" she can tell her email is delivered. perhaps failed/ will be used if she accidentally sent to bob1 instead-- or if bob3 deleted his account. but the former is unlikely in general because alice would probably copy her email to /home/alice1/mail/out/bob3/ every time. she would be most likely to get this wrong the first time, not afterwards. (if she knows a bob3 and a bob1 it could happen easily enough).
for very large groups of users, across different systems, why not just use email? indeed an interface to email (or just using email itself) is a possibility. but fmail is designed primarily for collaboration on a particular system, as an alternative to email. for other uses, other more complicated systems will possibly be preferred.
already we have outlined a ridiculously simple system for replacing email among users of a local filesystem. this system should work, it is as simple as possible, it only designs what needs to be designed and it allows simple clients that behave as the user wishes them to on top of a few basics that can be relied on from client to client or user to user.
its my preference to extend the system as follows:
the system is already clobber-resistant (mischief is technically possible by clobbering the date on someones unmoved read emails if someone sends the same file twice to the same person) and the filesystem handles date and time information.
the filename is a sha-256 hash of the file contents. i would like to extend the name with "-" as well as letters, ".", numbers and more dashes. i dont prefer to allow more than that, so users dont get saddled with files that are difficult to manage with scripts. but either way, this adds overhead (for checking or for handling) to the daemon.
it makes it much easier to track threads.
perhaps its simpler to just use mail(1). personally i find that solution so tedious, id rather invent something closer to the filesystem.
this has costs as well as advantages, but its made for people that like the filesystem and dont like mail(1).
whether this is implemented or not, it may inspire new tools for collaboration.