How to start at a different block height?

TL;DR

To start a SubQuery node synchronising from a non-zero height, all you have to do is to modify your project.yaml file and change the startBlock key.

For example:

Why NOT start from zero?

The main reason is that it can take a long time synchronising the entire Polkadot mainnet blockchain. Picking a start block such as 500,000 or 1,000,000 allows you to synchronise with the network faster.

The other reason is that if you know that you are only interested in say all transactions in the last 30 days, you can figure out the block height 30 days ago and start synchronising from there.

What are the drawbacks of not starting from zero?

The most obvious drawback will be that you won’t be able to query for data on the blockchain for blocks that you don’t have.

How to figure out the current block height?

Visit https://polkascan.io/ and pick your network. It will most probably be: https://polkascan.io/polkadot. You’ll then see the block height under “Finalised Block”.

Why is it not working?

The most common problem that you will encounter is that you did not delete your .data hidden folder.

If this is the first time running, you won’t have to do this because you won’t have a .data folder, but if you do, please remember to delete it.

Do I have to rebuild or codegen?

No. Because you are modifying the project.yaml file, which is essentially a configuration file, you will not have to rebuild or regenerate the typescript code. Just run:

docker-compose up

---

How to change the batch size?

TL;DR

Send the flag

--batch-size=50

in the command line to change the batch size from the default of 100 to anything that you want.

Introduction

For the astute amongst us, you may have seen the screenshot above where I am fetching 50 blocks at a time. (The default is 100). This can be customised with the extra command of –batch-size=<your number here>

Why change the batch size?

Using a smaller batch size can reduce memory usage and not leave users hanging for large queries. ie your application can be more responsive. However, more API calls are being made so if you are being charged on an I/O basis or if you have API limits somewhere in your chain, this could work to your disadvantage.

Where to change the batch size?

You need to add the –batch-size flag to the command line as an extra flag. If you are using docker, modify the docker-compose.yml file like so:

How do slots work?

Slots are the equivalent of deployment environments in a traditional software sense. Typically you have “localhost”, a development environment, a staging, maybe a QA, then a pre-prod and then production depending on your needs and/or budget.

At the minimum, one should have a development or “staging” environment and a production environment and that SubQuery has. SubQuery however uses the term “slots”.

Deploying to the staging slot is the same process as deploying to production slot. However, there are a few minor differences in terms of the options available.

In the staging slot, there are 2 additional options. a) Promote to production and b) delete.

The main benefit of the staging slot is that it allows you to prepare a new release of your SubQuery Project without exposing it publicly. The staging slot is not shown to the public in the Explorer and has a unique URL that is visible only to you. And of course, the seperate environment allows you to test your new code without affecting production.

Ref:

• https://www.youtube.com/watch?v=kwpwe3qPH4o
• https://subquery.medium.com/deployment-slots-are-here-subquery-projects-4fe2629f8858
• https://twitter.com/SubQueryNetwork/status/1400667147773247489