Technical Specification

The heart of the system

The core of the EveryBit platform consists of a username table managed by private keys, individual chains of content for each user, and a couple special blocks of content for user preferences and profile information.

Usernames

Every username has an entry in a Distributed Hash Table (DHT). Entries contains the following fields:
  • username
  • rootKey
  • adminKey
  • defaultKey
  • latest
  • updated
Usernames are formed with a string of alphanumeric characters. Once a username is created, it is permanently owned by whoever controls the private keys, subject only to the requirement that the user publish at least one piece of content per year. The updated field stores the date of the most recent update to the username record. Anytime new content is created, the user updates the latest field to point to their most recent content.

The owner of a username controls their sub-user space as well. For example, user .foo can create sub-users .foo.bar and .foo.fighters. There are several layers of keys in the record. New content is signed using the private key related to the defaultKey. The signature of a puff is checked against this same key to make sure it's a valid signature for that user. In order to add or modify a sub-user, the owner creates an update request and signs it with the private key related to their adminKey. The only way to change the adminKey or rootKey is to sign a message with the private key related to the rootKey. This is like a master key, and should be stored with the highest level of security. Cold storage is recommended. The default EveryBit client makes it easy to view QR codes for each of the private keys.

IMPORTANT: All signing of content and update messages happens on the client-side. Private keys should never be sent to a server.

For more about usernames, see the Username rollout section below.

Puffs

The main unit of content in the EveryBit platform is called a puff. It is structured as follows (required fields are indicated with an asterisk):
  • username*
  • routes
  • previous*
  • version*
  • payload
    • type*
    • content*
    • last
    • tags
    • time
    • parents
    • author
    • title
    • geo
    • copyright
    • zones
    • sig*
The identity of the person who created this puff is stored in username. The zones field serves to identify the intended recipients (if any) or to indicate that a puff is related to another user. It works like the @ sign in twitter.

Every piece of content has a unique id stored in the sig field. To generate this id, a user combines all of the fields of a puff (except the sig itself!) into a string and signs it using their private key. This signature serves as proof that the content really was created by the username listed. This also prevents accidental duplicate posts: two puffs that have the same sig also have the same content, and are in fact the same puff.

The version field corresponds to the version of the specification used by the puff. Right now the current version is 0.4.X. Until version 1.0 is reached, there may be changes to the structure of a puff. However, by specifying a version with each puff, it should be easier to deal with backward and forward compatibility issues.

The payload section of a puff contains the actual content, and meta-data about that content. The only two required fields are type and content. The others fields may or may not exist, and are subject to conventions about what they contain, instead of being specified directly. The type field is the same as the MIME type sent in HTTP headers. A single puff can only have one content type. This is vital part of the EveryBit platform -- it allows developers to treat each piece of content in the system as an atomic unit, and build a newer, much more powerful generation of RSS-like readers and search engines, ones which facilitate fine-grained aggregation (e.g. Show me the MP3's posted by .bach, any image from .ansel, and just the PGN's created by .fischer (but none of his annoying text posts!).

The payload section's content field contains the main content of the puff. This is always a string. For compound content types it is serialized in JSON format.

There are no rules about the other fields which can be included in payload, other than technical limitations to how they are specified (keys must be alphanumeric and less than 128 characters, values must be storable in JSON format).

In order to re-publish someone else's content, the entire puff is bundled up and put into the content field of the new puff, with type specified as "puff".

Note: The order of the top-level fields is important. For a puff to be verifiable the top-level fields (username, routes, previous, version, payload and sig) must conform to the order listed above. Field names within payload should conform to the order listed above, but that ordering is not currently required.

Profile and preferences

Every username has two special blocks of content associated with it. Both of them contain arbitrary key/value pairs related to that user. The profile block is for (generally public) information the user wishes to share about themselves. It could contain a field for their avatar or photo, information about where they work or how to contact them.

Design principles

The design of the EveryBit platform is driven by the following core beliefs:
  • Whenever possible, make decisions a convention and not a rule.
  • Provide good default values, then allow for customizability.
  • The client is king. Everything that can be done client-side, should be done client-side.
  • Separate content from interpretation.
  • Make it easy, beautiful, intuitive and fun.
  • Make it as secure and private as users want it to be.
  • Make it seem inevitable (because it is).

Multicontent

By convention, external dependencies should *not* be introduced into a puff. So if the type is HTML, then all JavaScript and CSS should be included directly in the content. Images can be included inline using the data:image/png specification.

How I learned to stop worrying and love immutability

Because of how puffs are constructed and chained, there is no way to edit a single puff without changing its id (signature), and disrupting the chain of content published thereafter. This is an intentional design decision. Here's why we do it this way:

When user replies to content, and embeds this parent puff's id in the new puff's parents array, they can be assured that so long as the puff they replied to still exists, it will not change (it's immutable). This creates an official, digitally signed conversation between the parties (that may be public or private). No party can claim to have written something they didn't, or change their words to make another user look bad. Because all discussion is contextual, we intentionally break all incoming connections to that users' more recent puffs as well.

This is an extreme thing to do, but we believe that the integrity of the system depends on it. We wish to encourage a culture where changing the content of a puff (especially one that has been around for a while and has generated a significant number of replies), is considered an extreme thing to do.

We wish to extend the cultural norms established by bloggers who pioneered the use of strikethrough to show that they edited a document for accuracy, usually based on feedback from others, and to fostering a culture of honesty and integrity in communication. Everyone makes mistakes, and we've found that there is a generally high level of tolerance to these. To encourage users to amend previous puffs instead of deleting and re-creating them, the default client shows all replies by the original author first. This way, the "OP" can post a reply amending their previous puff and know that this reply won't get buried by a torrent of angry replies that pick up on a mistake or omission.

Another way to mitigate the need to break your full content history just to correct a "bad" puff is to use different sub-usernames for different purposes. For example, if you create a EveryBit-enabled toaster that sends out a new puff any time the toaster leavin's are ready for harvesting, you could setup .username.toaster, or even .username.toaster.leavins, to publish these notifications. That way, if your toaster goes rogue and begins broadcasting bad information, you can roll back its chain of content without affecting your other streams of content.

Once the EveryBit platform is fully implemented, we imagine that developers will create tools to implement some kind of version control system with content merging (so that you could publish a "diff" puff to update a previous one). We encourage such development, so long as it's build upon an understanding of EveryBit's core strengths.