Blockchain is transforming various sectors and industries, including money, finance, banking, gaming, art, property, and many others. Those familiar with blockchain technology understand its fundamental role in trends such as cryptocurrency, dApps, Web3, and DeFi. Furthermore, the part that blockchain plays in cultural phenomenons like NFTs and decentralized gaming cannot be understated. Hence, if you want to join these sectors and create new dApps, you must learn how to index the blockchain.
A blockchain is a form of distributed ledger technology – a shared and immutable record of transactions in a network. Moreover, it eliminates the need for human trust in transactions. Thus, it lets the protocol implement the rules to automate this trust.
Since blockchain contains data, developers need to understand that the data needs indexing to make it useful in any Web3 build. Any new or aspiring blockchain developer should remember that knowing how to index the blockchain is key to harnessing its power.
Even to those seasoned in the best languages for blockchain development, such as JavaScript, Solidity, Rust, and others, indexing the blockchain can be a rather complicated process. It involves many steps, including spinning up your node(s) – one that can be made simple by using IaaS (infrastructure as a service) products like Moralis Speedy Nodes.
This guide will show you the tools to use as you explore how to index the blockchain. You can do it from scratch, using multiple platforms. You can also use node providers such as Infura or Infura alternatives. Alternatively, you can cut through the complexity and use Moralis to handle the intricate backend. Hence, using Moralis will save you precious time and effort. Plus, it gives your dApp the edge of being first to market!
How to Index the Blockchain – Video Walkthrough
If you want to go directly to an overview of how to index the blockchain that focuses on two platforms: The Graph and Moralis – check out the following video from the Moralis Web3 YouTube channel:
What is Blockchain Indexing?
To understand how to index the blockchain, you need to grasp the technology first. Additionally, you need to understand its applications. Moreover, for a broader idea of these applications, read the Moralis blog article called “Use Cases for Solidity Smart Contracts”.
What is Blockchain?
Blockchain is a distributed and shared ledger in a network. It records transactions and broadcasts them in an open, transparent, and immutable way. Furthermore, it allows you to create and track a digital asset (a coin or a token) in a network, paving the way for borderless trading and value exchange.
Why is blockchain relevant today? Blockchain promotes decentralization by letting protocols do the work of authority figures and institutions. Therefore, it plays a role in disintermediating processes.
What is a blockchain made of? The first component is the distributed ledger itself, meaning that everyone in the network has a copy. The property of immutability makes up the second component. Cryptographic hashes enable immutability because such hashes cannot be reverse-engineered, at least not for a long time and not without considerable computing power.
The third component is smart contracts. However, what are smart contracts? Smart contracts determine the rules and features in a blockchain protocol. Furthermore, they are coded in Solidity or similar high-level languages.
Why Learn How to Index the Blockchain?
It is one thing to have distributed and immutable data and another to make it useful to a developer trying to build a new platform or create an Ethereum dApp. Therefore, you need to learn how to index the blockchain.
With the massive amount of data available, which blockchain tends to store in a “scattered” way, you need a system of filtering or locating the data you need for your platform. In other words, you need the ability to query the blockchain data to use in your dApp.
Blockchain Indexing Alternatives
As you explore how to index the blockchain, you will find two popular alternatives: The Graph and Moralis. This guide examines the differences between the two. Thus, it presents the processes you need to go through when using either platform as you navigate how to index the blockchain.
Additionally, as you compare the various tools that teach you how to index the blockchain, you need to consider the following:
- The time frame for developing the dApp, platform, or product.
- Your skill set and stack, including your understanding of how to create smart contracts.
- The amount of complexity you are willing to go through to achieve proper indexing.
- Type of project and the features needed.
- Your level of experience when it comes to running Web3 backend infrastructure.
Blockchain indexing protocols such as The Graph and Moralis offer ways to query data that would otherwise be difficult to query directly from the blockchain. You need these tools to ensure that your indexing capability goes beyond basic read operations.
Understanding how to index the blockchain allows you to do more than interact directly with the contract. Moreover, it lets you harness valuable information associated with complex or advanced operations. Search, aggregation, relationships, or non-trivial filtering are made possible by tools like The Graph and Moralis.
How to Index the Blockchain – It’s Difficult
There is no way to “gloss over” it – the process of understanding how to index the blockchain is quite challenging. Moreover, it gets complicated even for advanced developers. It involves a lot of time, dedication, and meticulous effort to pull off.
The inherent properties of the blockchain, such as finality, chain rearrangement, and uncalled blocks, are a double-edged sword: they make retrieval of accurate querying results hard. However, tools that simplify how to index the blockchain, such as The Graph and Moralis, can ease some of the pain points.
How to Index the Blockchain Using The Graph
The Graph is a popular platform that addresses how to index the blockchain. The Graph implements an indexing protocol that promotes accessibility of dApps through public and open APIs, which it calls “subgraphs”. This indexing tool, just like any indexing tool for traditional databases, is capable of locating and retrieving data on Ethereum.
Network Participants and Incentives
The Graph’s network has delegators, indexers, and curators. These provide data to Web3 apps or dApps. Moreover, such participants also provide network services powered by GRT or The Graph tokens (ERC-20 tokens) as incentives and a means of securing the network.
The Subgraph Manifest
When attempting the process of how to index the blockchain using The Graph, you need to remember a key term: the subgraph manifest. This is the set of subgraph descriptions that lead the protocol to learn how to index the blockchain data on Ethereum.
Thus, the subgraph manifest directs the protocol to the smart contracts of interest for a particular subgraph. In addition, it teaches The Graph the process of mapping event data versus the data that needs storage in its database.
To get a better overview of how a subgraph manifest directs the flow of data after its deployment, you can view this schema from The Graph’s site:
This sample flow can be summarized as follows:
- A dApp contributes its data to the Ethereum protocol using a smart contract transaction.
- While conducting the transaction, the smart contract will generate one or more events.
- The Graph’s node in the middle constantly scans Ethereum for new blocks and the data they provide for the subgraph.
- Afterward, The Graph’s node searches such blocks for the events relating to a particular subgraph. Moreover, it executes the mapping handlers that you previously specified.
- The dApp utilizes the GraphQL endpoint of The Graph’s node to query the node for the data indexed from the blockchain. The Graph’s node converts these GraphQL queries to SQL queries for its underlying data store to retrieve the necessary data.
- After this, the dApp now presents the data to the end-users. This happens through a rich UI, which can then create new transactions in the protocol.
- The entire cycle repeats.
How to Index the Blockchain Demo
To see how these steps are performed in a practical yet simple roulette betting demo dApp, make sure to check out the video above. Start at the beginning for a proper introduction. However, proceed to the 5:38 timestamp if you want to dive in immediately into using The Graph.
The demo teaches you how to create a subgraph on The Graph. You will go through The Graph’s process of how to index the blockchain. This involves deploying smart contracts as part of a simple game application (in this case, a betting game). A subgraph is created on Ethereum’s Rinkeby testnet (a similar process works for a local dev chain such as Ganache). Afterward, you will deploy and publish the Rinkeby subgraph and configure the subgraph “yaml” file, mappings, and schema.
Next, you’ll do some queries on the playground to pull in various data previously indexed from a smart contract.
Through this demo, you catch a holistic glimpse of the process. You can pinpoint the challenges along the way. Plus, you will learn that it takes several attempts and adjustments to get things working perfectly.
Indexing the Blockchain – The Output
The subgraph output should look like the image above, where you can query the smart contract on the blockchain and get in the last five results from the deployed app smart contract.
Pull the Indexed Data to the Frontend
Ultimately, the goal is to bring all these aspects together – to aggregate data and pull them to the frontend. Moreover, the tutorial reminds you to keep referring to The Graph’s website to guide you through the meticulous process. Additionally, the steps are quite similar if you want to do this on a local blockchain, and the instructions are available on The Graph’s site.
The Faster Indexing Protocol Alternative – Moralis
In studying how to index the blockchain, you might need a quicker and simpler tool to get things done. Moralis enacts the same process of how to index the blockchain in an easier format, as it does most of the heavy lifting on the backend. Therefore, Moralis becomes a faster blockchain indexing protocol and establishes itself as the premier choice for blockchain developers.
Here are the steps to utilize Moralis:
Step 1 – Create an Account on Moralis
Step 2 – Create a New Server
After signing up, add a Moralis instance. To do so, click on “+ Create new Server” in the upper-right, and select “Testnet Server”:
Then, choose “Eth (Rinkeby)” as your chain, name the server, select your region and click on “Add Instance”. The latter will spin up the server, which takes a few minutes.
Step 3 – Get the Info
Now, go to VSC (Visual Studio Code). You’ll need to get the necessary information from the smart contract, such as the smart contract address and the “bet” event.
You will find detailed instructions on this at 27:13 of the video. The video gives pointers on what info to keep (such as the type of data you need to query), discard, and how to tweak the code. Use a notepad or similar to tidy the code up to remove the things you don’t need.
Get the “bet” Event for Your dApp
Next, you need the ABI for the “bet” event. Copy the ABI, as indicated in the image below, and add it to your notepad.
Step 4 – Go Back to Your Moralis Server
Click on the “View Details” button. As such, you’ll find a pop-up window with your server information. Now, click on “Sync” and then “Sync and Watch Contract Events”:
Afterward, a pop-up window will appear that requires you to enter some information:
Proceed by filling in the information. Enter your description and check the box that asks if you should sync historical data (yes, you should). Under the topic, enter the “bet” event. Paste the ABI of the “bet” event, and then add the smart contract address. Finally, add a name to the “TableName” section. In this tutorial, we named the “TableName” section “Spins”. Once done, click on “Confirm”.
Close the screen and go to your Moralis dashboard, which you can access by clicking on the “Dashboard” button at the lower part of your server bar.
Upon viewing the dashboard, you will find the “Spins” table. If it’s working correctly, you’ll find that Moralis has already pulled in the data related to your previous inputs. You’ll find, in this case, the bet amounts, side number, block number, the win “Boolean”, and the confirmation data (whether the block was mined). Plus, the address of the user who did the spin is found under the “user String” column.
From here, you can start querying the data in your database. You can now build a dApp that uses the details for your server. You can create an “index.html” file, add some JavaScript to initialize Moralis, and then connect to the Moralis SDK using just a few lines of code as shown here:
The Moralis SDK
Go back to “View Details”, get the server URL and application ID, and add it to your “index.html” file. In turn, you’ll initialize Moralis and add its functionality. Right-click on it, and run it using VSC’s Live Server extension. The remaining steps, such as user authentication and querying, are outlined at 30:43 of the video.
How to Index the Blockchain – Summary
As you weigh alternatives for indexing the blockchain, you must consider multiple factors. Among these are time, skill, experience, convenience, features of your dApp, and your level of familiarity with smart contracts and node infrastructure.
The Graph is a powerful and popular platform for blockchain indexing. It has many excellent features and allows more avenues for a high degree of configuration and customization. It is suitable if you want to run your server and infrastructure. However, the setup process can get quite complicated. Furthermore, it may prove impractical for many types of projects. Also, you will find that this setup hits the node quite heavily, which will lead you to jump from one node provider to another.
Moralis, the simpler and markedly quicker alternative with its powerful Web3 SDK, lets you do the same thing with minimal coding and effort. At a minimum, you’ll need JavaScript skills. What’s more, Moralis is cross-chain by default. It supports multiple blockchains such as Ethereum, BSC (Binance Smart Chain), Polygon, Avalanche, and Arbitrum, and it streamlines the complicated node setup process typically required for cross-chain dApps.
Moralis is the quickest route from A to B, leading you from prototype to finished product in a matter of days. Moreover, as the ultimate Web3 platform, Moralis has all the features you need to simplify how to index the blockchain. As such, you can focus on the user and market-related aspects: building a beautiful frontend and seizing that first-mover advantage for your dApp.
