Skip to main content

The Spork CLI tool

· 6 min read
Paul Frazee

Look out, mr elephant!

Alternative title: What's the best way to eat an elephant?

Suppose you set up a server on your home network. Unless you configure DDNS or a static IP, your server can only host apps for your LAN. It's isolated, and the isolation means it can't do the kind of collaborative and social tasks we expect on the Internet.

Now what if your friend also had a home server, and we could get them to connect and send messages, share files, sync databases, etc? Now we're not isolated anymore. Now we can run social and collaborative applications which typically run on commercial clouds.

This is the idea for the Atek Cloud project. It will be something like a personal PaaS that’s consumer-friendly. It will use p2p and web3 tech to make the apps socially connective with each other.

Web 3 network

There's only one problem: how does it work? Not in a broad sense, but, like, in the actual execution. How do we implement this thing?

Does Atek run Docker images, or some new JS serverless runtime? Do we go nuts on a VM-based sandbox, perhaps with Firecracker? Does Atek implement a users API with SSO, or leave authentication and perms to the apps? Should it provide a novel peer-to-peer database that all apps share, or should data be isolated to each application? Is service discovery between the apps important? What about device clusters? The more we've talked to folks, the more questions we've uncovered about the implementation.

All this means that the Atek platform is an elephant-sized task, and you don't eat an elephant all at once. You eat an elephant one bite at a time.

A complex system that works is invariably found to have evolved from a simple system that works. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work.

― John Gall, The Systems Bible

Sporking the elephant#

Spork is a command-line tool that implements the parts of Atek that we're sure about. We called it Spork because it's a fun tool that does multiple things. (The metaphorical relevance to elephant-eating is coincidental.)

$ npm install -g @atek-cloud/spork

At the moment what Spork does is peer-to-peer tunneling. You can, for instance, expose port 8080 as a p2p socket with the following command:

$ spork bind -p 8080Created temporary keypair, public key: whattzzuu5drxwdwi6xbijjf7yt56l5adzht7j7kjvfped7amova
======================Spork powers ACTIVATED
  - Mode: Reverse proxy  - Listening on whattzzuu5drxwdwi6xbijjf7yt56l5adzht7j7kjvfped7amova  - Proxying all traffic to localhost:8080======================
If you're running an HTTP server, you can share the following link:
  https://whattzzuu5drxwdwi6xbijjf7yt56l5adzht7j7kjvfped7amova.atek.app/
This public gateway will tunnel directly to your spork!

Now somebody else can spork bind {pubkey} to create the other end of that tunnel on their local device. If they do so, they get an end-to-end encrypted tunnel which handles hole-punching through a distributed network. There's an elephant bite handled.

There are some pretty obvious use-cases for spork bind. Sharing Web apps are obvious, maybe when you're doing dev and need to share the link. We managed to spork an SSH session between Texas and Virginia (through Starlink!). Some folks on the discord sporked a Minecraft game. It currently exposes TCP socket only, but anything that goes through a single TCP connection should work.

The public gateway#

You'll notice a link gets dropped in the spork bind output that looks like https://{pubkey}.atek.app. That's a public gateway similar to the ones used by IPFS but for the p2p tunnel. Using that you can share a link your app without the recipient having to install spork themselves. You lose the end-to-end privacy and it's HTTP only, but you gain some convenience.

We felt like a gateway was important to help with the asymmetry of adoption and we'll keep it running until we go broke. It's pretty easy to run your own though with spork gateway.

You could probably CNAME a domain name to your {pubkey}.atek.app address. I haven't tried this and wouldn't suggest it, but if you did I wouldn't be mad.

Next bites of elephant#

The next things we'll do with Spork will be convenience commands. Perhaps a quick "serve this folder as a website" command. Maybe an SSH tunnel command. SFTP, perhaps? Anything that makes life easier.

After that, I suspect we'll try to simplify the key-sharing flows. This might include magic-wormhole-style tools and local address-books, which gets us into identity systems and service-discovery.

The Hypercore Protocol is stabilizing some new distributed data structures. I can easily see Spork adopting Hypercore's key-value database (Hyperbee) into its toolset and then building upon that. Maybe to implement registries? Maybe to distribute software builds? Maybe for static websites? Some folks are even toying with smart-contracts using the append-only log.

From there, perhaps a chat command. Perhaps mail. Perhaps a simple way to run nodejs scripts as p2p services and automatically broadcast them to friends' service registries.

The point being, we have ideas but no concrete plans — and that's what Spork is for! We're going to keep iterating on the Atek stack by putting toys into Spork. Each toy gets us closer to understanding how Atek's platform should be designed. Next thing you know, there won't be any elephant left.

Credit to the Hypercore protocol#

I have to be clear that most of Spork's magicks come from the incredible Hypercore Protocol team. Spork is really a small wrapper around Hyperswarm, their networking layer.

It's important to me to give credit to Mathias, Andrew, David, and the rest of that team because they're the ones who deserve it, and also because otherwise people will start asking me questions about Spork's cryptography that I can't begin to answer.

Hop on the Hypercore Protocol Discord if you want to dig more into the work they're doing.

Links#

Paul

We at Atek do not condone the eating of elephants except in the metaphorical sense.