IPFS Production Configuration

0
142

Near future, our team will use IPFS for production service so I investigated lots about the configuration of it. I’m going to share my learned knowledge.

1. Preparation

At first, export “IPFS_PATH”.

export IPFS_PATH=/data/ipfsrep

The IPFS repository is created to this path. The default path is your $HOME. For later convenience, I highly recommend setting this path to the proper location.

2. Repository initialization

Initialize repository with “— profile server” option.

ipfs init —- profile server

This option optimizes IPFS configuration for a data center as disabling host and content discovery in local networks. See official document

If you are running on a server in a data center, you should initialize IPFS with the server profile. This will prevent IPFS from creating a lot of data center-internal traffic trying to discover local nodes

3. Edit configuration file

IPFS provide us command line “config” option, though I think it’s easy to edit the configuration file manually. The configuration file is located right under “IPFS_PATH”

vi $IPFS_PATH/config

From the beginning, edit “Addresses.API” and “Addresses.Gateway”. API is writable though Gateway is read-only. When you set Gateway as “0.0.0.0”, Gateway is open to the internet.

"Addresses": {
"API": "/ip4/127.0.0.1/tcp/5001",
"Announce": [],
"Gateway": "/ip4/0.0.0.0/tcp/8080",

Next, edit “Datastore.StorageMax”. This is an upper limit for the size of the IPFS repository’s datastore. The default values is “10GB”, but you may need more.

"StorageMax": "100GB"

Next, follow official instruction. Basically, this site is for the cluster, but helpful for single as well.

Increase the Swarm.ConnMgr.HighWater (maximum number of connections) and reduce GracePeriod to 20s.

“HighWater” is the number of connections that, when exceeded, will trigger a connection GC operation. And “GracePeriod” is a time duration that new connections are immune from being closed by the connection manager.

Increase Datastore.BloomFilterSize according to your repo size (in bytes): 1048576 (1MB) is a good value

“BloomFilterSize” is a number representing the size in bytes of the blockstore’s bloom filter.

The IPFS_FD_MAX environment variable controls the FD ulimit value that go-ipfs sets for itself. Depending on your Highwater value, you may want to increase it to 4096.

FD means file descriptor. “IPFS_FD_MAX” specify open file descriptor limit.
According this code, the default max FD is “8192” so that it’s better to set it as “4096".

Finally, if you use as Gateway, edit “Gateway.HTTPHeaders”. By default, CORS is enabled. We can specify “Access-Control-Allow-Headers” or “Access-Control-Allow-Origin” here. If you never run Gateway as writable, It’s better to remove “POST” and “PUT” methods from “Access-Control-Allow-Methods”.

"Gateway": {
"APICommands": [],
"HTTPHeaders": {
"Access-Control-Allow-Headers": [
"X-Requested-With",
"Range",
"User-Agent"
],
"Access-Control-Allow-Methods": [
"GET"
],
"Access-Control-Allow-Origin": [
"*"
]
},

4. Run background

For development purpose, you can start IPFS daemon background with following a simple command.

ipfs daemon > $IPFS_PATH/ipfs-`date +%Y%m%d%H%M%S`.log &

Then, stop like this command.

ps ax | grep ipfs | head -n 1 | awk ‘{print $1}’ | xargs kill

For production purpose, it’s better to use systemd. To create a system user and change the owner of “$IPFS_PATH”. Then, create a configuration file named “ipfs.service”.

adduser — system — group ipfs
chown -R ipfs:ipfs $IPFS_PATH
vi /lib/systemd/system/ipfs.service

As for configuration, this GitHub issue is very helpful.

Show you example configuration as bellow. Please change “IPFS_PATH” depending on your environment.

[Unit]
Description=IPFS daemon
After=network.target
Requires=network.target
[Service]
Type=simple
User=ipfs
RestartSec=1
Restart=always
Environment=IPFS_PATH=/data/ipfsrep
Environment=IPFS_FD_MAX=4096
ExecStart=/usr/local/bin/ipfs daemon
[Install]
WantedBy=multiuser.target

5. Support HTTPS

IPFS doesn’t support HTTPS so we need to prepare a proxy server or a load balancer for SSL offloading purpose.

See this blog post for the proxy setting.

See my previous blog post for azure load balancer setting.

For your information, the default API port is “5001” and the default Gateway port is “8080”.

6. Stop swarming

IPFS is trying to connect other nodes depending on a bootstrap peer list. You can see it by the following command.

ipfs bootstrap list

If you use IPFS as readonly, I think it’s better to stop swarming for network fee-saving purpose. The default port is “4001”. Please keep mind you need to close Inbound, additionally Outbound as well. IPFS node tries to connect from themselves so we need to close Outbound.

7. Backup and Restore

You just back up all files of under “$IPFS_PATH”. You can restore all data from under “$IPFS_PATH” files.

8. Cluster Setting

Unfortunately, our team doesn’t use cluster so that please see the official site or helpful blog post.

9. Tips for storing hash to Ethereum

For someone who plans to store IPFS hash to Ethereum, Tell you tips to save gas cost. Don’t store IPFS hash as string type. Store it as bytes32 type. IPFS hash string consumes more memory than bytes32 so that It’s costly.

IPFS hash is base58 encoded value. So we just decode it to obtain bytes32.

See Origin Protocol code as a code example. And see my previous blog for more tips saving gas cost.

Summary

IPFS production configuration information is less so it may not be enough as the above steps. Please keep watching updates

If you run a small service, these steps may be too much. In this case, just initialize repository with “ — profile server” option. IPFS optimize configuration.

I hope to be helpful.


IPFS Production Configuration was originally published in Coinmonks on Medium, where people are continuing the conversation by highlighting and responding to this story.