The Moralis Research Team is very excited to share this new publication with you! Every week we’ll deep dive into pressing blockchain programming problems, and show you how Moralis makes solving them simple. Read on for
Our goal at Moralis is to support you and your projects. Think of us as your “wingman” as we build the future of finance together.
Up This Week: IPFS
This week, we look at IPFS file hosting, and how to use Moralis to effectively leverage this excellent storage technology.
What is IFPS?
The InterPlanetary File System (IPFS) is a protocol and peer-to-peer network for storing and sharing data; a distributed file system.
It’s hailed as the replacement to the standard web protocol HTTP and HTTPS (which reference locations).
When an internet user gives a domain name to their browser (example: http://www.google.com), that domain points to an IP address of a server that hosts the content.
Where is IPFS Data Stored and How do I Find it?
Each piece of content that the user wants to publish is hashed, with some metadata. That hash creates a unique identifier called a CID or ‘Content ID’. IPFS content is decentralized and distributed among peers, the information can’t just be taken down.
It’s everywhere, and it’s immutable, meaning users cannot change the content and still retain the same Content ID (unless it’s IPNS).
IPFS participants don’t need a fixed location to store IPFS data, they only need the Content ID and it goes across the network to find the data. The Content ID is a hash, so it doesn’t care where the content (data) lives (IPNS – which is similar to IPFS – IS location-based).
How does IPFS work?
The simple answer is ‘distributed hash tables’ or DHTs. They map the Content ID to peer addresses (similar to public IP addresses and ports), so basically there are a bunch of DHT servers that serve these DHTs to clients that connect to the servers.
They relay messages saying “this Content ID lives in this peer”, or “this Content ID lives in this IP address”. So if we went to an address like ipfs://x, this is asking for the CID of ‘x’ – it will query the DHT table to retrieve all peers that have that particular CID, and then connect to those peers to allow the user to consume the content.
Consuming IPFS Content
The IPFS client consults a local DHT table to see where the content is located, and returns a collection of IP addresses and ports as normal. Maybe the local DHT table has nothing, it wasn’t updated, so nothing came back.
In this case, you’ll start asking your connected peers “Does anybody know where to find this content?”
Let’s say you do get the IP addresses and port (the locations where the content resides), then the clients connect to some or all of the peers, and can start downloading the content.
How is Content Downloaded?
It’s a torrent (peer-to-peer) protocol. The client downloads chunks of the content from each peer, speeding up the process. The protocol breaks the content into specific chunks, each with a specific hash, and the client requests the individual pieces from peers.
You start opening multiple connections to multiple peers and you start downloading the content in parallel. Once the client has the content, it also updates its local DHT table with the CID. The user can decide whether to host or ‘leech’ (the concept of consuming only – you basically download the content and then kill the seed). The Brave browser contains a feature that enables or disables hosting.
How do You Find Peers?
When you first install an IPFS client the IPFS protocol gives you a few of your peers, that’s how you get into the network.
The first step is to connect to its local peers through ‘bootstrapping’ and once they begin hosting content, they update peers of their DHT table through a fairly complex process. Read up on Kademlia for more.
What if You Change Content?
The content is immutable. If you make a change to any of the content you’ll need to generate a new content id and you have to update the DHT table and reinform the users and it’s not so easy. One solution to this is, to hash the public key of the user instead.
This way the public key will be mapped to a name server, and the public key hash will point to the content. So people will just keep consuming your public key hash instead of a content id. This is IPNS (InterPlanetary NameServer). IPNS is mutable and can be changed.
How Can I Delete the Content?
If you are the only node hosting that content, then you can unpin it, which will remove the entry from your local DHT. However once another node downloads that content, you cannot remove it anymore – something to be aware of.
There are hungry nodes with triggers to download content automatically, so if you upload to IPFS, it’s highly likely that the content can no longer be removed.
How Do I Publish to IPFS?
If you have content to publish and you want to share on IPFS, your content is first hashed into a unique content id. Your local DHT table will update your CID to your IP address and you will start as the only one hosting that content.
The content is then propagated across all the connected peers – assuming you want it publicly accessible. Only the DHT table is propagated, not the content itself. People searching for your CID do not host the content by default. You are the only owner until a user downloads the content.
You could use a command-line package manager to install IPFS. For example ‘brew’ on a mac. Once you install IPFS, you can spin up a node to host content and read directly from the network. It will also show you all the peers you have. You can even consume content directly from the terminal, if desired. Alternatively, you could create content from the command line and access it using brave.
Or, you could use MORALIS…
The MORALIS site has detailed documentation on IPFS which will help you use it with your dApp. You can also check out these useful videos on the topic on the MORALIS YouTube Channel:
How to Save ‘Objects’ Like Metadata to IPFS
How to Save ‘Files’ to IPFS
How to Combine ‘Files’ and ‘Objects’ and Upload Them to IPFS
Thanks for reading! We hope you found this week’s Moralis Magazine useful.
Keep BUIDLING!
See you next time 💚
The Moralis Research Team
