Chia Automation Scripts – Plotting and Farming

Chia Automation Scripts – Plotting and Farming

[Advertising] For quite some time now I have been offering shell scripts for use with Chia for sale. Millionminer.de – a specialized trade for cryptominer – has now agreed to make a small gift to my readers and made the free publication of the following scripts for Chia Network possible.

[Post] The shell scripts for Chia include the following content:

  • Install Farmer
  • Install plotter
  • Update Farmer
  • Automatically update the farmer
  • Fetch plots from the plotter automatically
  • Distribute plots on the farmer automatically
  • Example of automatic plotting
  • Example for aggressive automatic plotting
  • General notes about the scripts

Additionally I commented the scripts in english inside the script. The scripts are designed for Linux Debian.

Install the Farmer


#dont forget to format and mount your drives
#update the system and install some tools and chia as root
#location: /home/
apt-get update
apt-get -y upgrade
apt-get -y install screen
apt-get -y install lsb-release
apt-get -y install git
apt-get -y install iotop
apt-get -y install rsync
apt-get -y install sshpass
apt-get -y install smartmontools
cd /home/
git clone https://github.com/Chia-Network/chia-blockchain.git -b latest --recurse-submodules
cd chia-blockchain
chmod 777 install.sh
echo 'install as root'
sed -i 's/sudo //g' /home/chia-blockchain/install.sh
sed -i '$a. ./activate' /home/chia-blockchain/install.sh
sed -i '$achia init' /home/chia-blockchain/install.sh
sed -i '$achia version' /home/chia-blockchain/install.sh
sed -i '$agit status' /home/chia-blockchain/install.sh
sed -i '$aecho lautenbacher.ch' /home/chia-blockchain/install.sh
echo 'disable strict host key checking'
sed -i '$aStrictHostKeyChecking no' /etc/ssh/ssh_config
./install.sh
echo 'add the plot directories to the farm'
echo 'example: chia plots add -d /hdd/1/'
echo 'example: chia plots add -d /hdd/2/'
echo 'chia keys add - dont forget to add the keys'

Install the Plotter


#dont forget to format and mount your drives
#update the system and install some tools and chia as root
#location: /home/
apt-get update
apt-get -y upgrade
apt-get -y install screen
apt-get -y install lsb-release
apt-get -y install git
apt-get -y install iotop
apt-get -y install rsync
apt-get -y install sshpass
apt-get -y install smartmontools
cd /home/
git clone https://github.com/Chia-Network/chia-blockchain.git -b latest --recurse-submodules
cd chia-blockchain
chmod 777 install.sh
#install as root
sed -i 's/sudo //g' /home/chia-blockchain/install.sh
sed -i '$a. ./activate' /home/chia-blockchain/install.sh
sed -i '$achia init' /home/chia-blockchain/install.sh
sed -i '$aecho lautenbacher.ch' /home/chia-blockchain/install.sh
#disable strict host key checking
sed -i '$aStrictHostKeyChecking no' /etc/ssh/ssh_config
./install.sh
#there is no need to add keys
#dont forget to make tmp and plot directories on the SSDs for the plotting

Update the Farmer

#updatescript for the farmer
#stop the farmer - update everything as root - restart the farmer
#location: /home/
apt-get update
apt-get -y upgrade
cd /home/chia-blockchain
. ./activate
chia stop -d all
deactivate
git fetch
git checkout latest
git reset --hard FETCH_HEAD
git fetch
git checkout latest
chmod 777 install.sh
sed -i 's/sudo //g' /home/chia-blockchain/install.sh
sed -i '$a. ./activate' /home/chia-blockchain/install.sh
sed -i '$achia init' /home/chia-blockchain/install.sh
sed -i '$achia start farmer' /home/chia-blockchain/install.sh
sed -i '$achia version' /home/chia-blockchain/install.sh
sed -i '$agit status' /home/chia-blockchain/install.sh
sed -i '$aecho lautenbacher.ch' /home/chia-blockchain/install.sh
./install.sh

Update the Farmer automatically

#automatic updatescript for the farmer (1 time per day)
#stop the farmer - update everything as root - restart the farmer
#location: /home/4-updatefarmer-automatic.sh
apt-get update
apt-get -y upgrade
cd /home/chia-blockchain
. ./activate
chia stop -d all
deactivate
git fetch
git checkout latest
git reset --hard FETCH_HEAD
git fetch
git checkout latest
chmod 777 install.sh
sed -i 's/sudo //g' /home/chia-blockchain/install.sh
sed -i '$a. ./activate' /home/chia-blockchain/install.sh
sed -i '$achia init' /home/chia-blockchain/install.sh
sed -i '$achia start farmer' /home/chia-blockchain/install.sh
sed -i '$achia version' /home/chia-blockchain/install.sh
sed -i '$agit status' /home/chia-blockchain/install.sh
sed -i '$aecho lautenbacher.ch' /home/chia-blockchain/install.sh
sed -i '$aecho waiting 1 day' /home/chia-blockchain/install.sh
sed -i '$asleep 86400' /home/chia-blockchain/install.sh
sed -i '$a/home/4-updatefarmer-automatic.sh' /home/chia-blockchain/install.sh
./install.sh

Get the Plots from the Plotter automatically

#rcp is also possible instead of rsync
#if you not use rsync please make sure you exclude all possible temporary filenames like .temp or other file endings
#location: /home/

echo "Start Transfer"

space_free=$(df | grep -i /temphdd/1 | awk '{print $4}')
if [ $space_free -ge 115343301 ]
then
sshpass -p "password" rsync --remove-source-files --progress --recursive -e 'ssh -p22 -T -o Compression=no -x' --include "*/" --include="*.plot" --exclude="*" [email protected]:/tempstorageonplotter/ /temphdd/1 
else
echo "No Space left on TempHDD1"
fi

space_free=$(df | grep -i /temphdd/2 | awk '{print $4}')
if [ $space_free -ge 115343301 ]
then
sshpass -p "password" rsync --remove-source-files --progress --recursive -e 'ssh -p22 -T -o Compression=no -x' --include "*/" --include="*.plot" --exclude="*" [email protected]:/tempstorageonplotter/ /temphdd/2
else
echo "No Space left on TempHDD2"
fi

sleep 5m

./farmer-gettheplots.sh

Move the Plots automatically

#location: /home/

echo "Start moving"


space_free=$(df | grep -i /hdd/1 | awk '{print $4}')
if [ $space_free -ge 115343301 ]
then
rsync --remove-source-files --progress  --recursive --include "*/"  --include="*.plot" --exclude="*" /temphdd/1/ /hdd/1/
rsync --remove-source-files --progress  --recursive --include "*/"  --include="*.plot" --exclude="*" /temphdd/2/ /hdd/1/
else
echo "No Space on HDD1"
fi

space_free=$(df | grep -i /hdd/2 | awk '{print $4}')
if [ $space_free -ge 115343301 ]
then
rsync --remove-source-files --progress  --recursive --include "*/"  --include="*.plot" --exclude="*" /temphdd/1/ /hdd/2/
rsync --remove-source-files --progress  --recursive --include "*/"  --include="*.plot" --exclude="*" /temphdd/2/ /hdd/2/
else
echo "No Space on HDD2"
fi


space_free=$(df | grep -i /hdd/3 | awk '{print $4}')
if [ $space_free -ge 115343301 ]
then
rsync --remove-source-files --progress  --recursive --include "*/"  --include="*.plot" --exclude="*" /temphdd/1/ /hdd/3/
rsync --remove-source-files --progress  --recursive --include "*/"  --include="*.plot" --exclude="*" /temphdd/2/ /hdd/3/
else
echo "No Space on HDD3"
fi

#if you have more final hard drives please extend the list and dont forget to add all disks to the farm
#do not forget slashes behind the directories

sleep 5m
./farmer-movetheplots.sh

Example for automatic Plotting

# stagger the jobs with sleep at first run (1 to 2 hours)
# 3600 = 3600 seconds
# you need to make an separate screen session and sh script for each job
#location: /home/chia-blockchain/

sleep 3600
. ./activate
for (( ; ; ))
do
   echo "infinite loops [ hit CTRL+C to stop]"
#start the plotting with 4 cores
chia plots create -f farmerpublickey -p poolpulickey -k 32 -r 4 -n 1 -t /ssd/1/temp/ -d /ssd/1/plot/
#copy the final plot to the fast temp storage on the same machine
rsync --remove-source-files --progress --recursive --include "*/"  --include="*.plot" --exclude="*" /ssd/1/plot/ /tempstorageonplotter/
done

Example for aggressive Plotting – Part 1

# stagger the jobs with sleep at first run (1 to 2 hours)
#location: /home/chia-blockchain/
sleep 3600
. ./activate
for (( ; ; ))
do
   echo "infinite loops [ hit CTRL+C to stop]"
#start the plotting with 4 cores
chia plots create -f farmerpublickey -p poolpulickey -k 32 -r 4 -n 1 -t /ssd/1/temp/ -d /ssd/1/plot/
#copy the final plots will be done within an extra job in the aggressive version
done

Example for aggressive Plotting – Part 2

#copy the final plot to the fast temp storage on the same machine
#location: /home/chia-blockchain/

for (( ; ; ))
do
   echo "infinite loops [ hit CTRL+C to stop]"
rsync --remove-source-files --progress --recursive --include "*/"  --include="*.plot" --exclude="*" /ssd/1/plot/ /tempstorageonplotter/
rsync --remove-source-files --progress --recursive --include "*/"  --include="*.plot" --exclude="*" /ssd/2/plot/ /tempstorageonplotter/
rsync --remove-source-files --progress --recursive --include "*/"  --include="*.plot" --exclude="*" /ssd/3/plot/ /tempstorageonplotter/
sleep 5m
done

Additional Information

With file 1 you can install the farmer
With file 2 you can install the plotter
With file 3 you can update the farmer if there is a new release out

All scripts need to be executable (chmod 777 ./file.sh). The recommended location for each script is inside a comment like #localation: /path/

All scripts are intended to run within a screen session (except manual installation und manual update). To start a screen session type “screen”, you can leave the session without terminating it with (CTRL+SHIFT)+D

You only need to add the keys on the farmer
On the plotter you can create new plots with the farmer public and pool public key
At this point there is also no need to perform updates on the plotter.

You can see both keys on the farmer with “chia keys show”
please be aware that you have to switch into the chia directory and type “. ./activate” for being able to use “chia” commands

With the cmd “chia farm summary” you see information about the farm
with the cmd “chia wallet show” you see information about the wallet (first time use press “S” if asked)

plotjob1-exampleforplotter.sh

Please create the necessary directories on the ssds/nvmes (temp / plot).
It’s not really necessary to make different temp and plot dirs but I still recommend it to you (temp1, temp2, plot1, plot2).

On the plotter you can make jobs for plotting, the example is the plotjob file.
You need at least 240 GB temp space and around 102 GB for the final plot for one job

Note: If you want to plot “more aggressive” you also can split the infinite plotting and the moving of the final plots in two seperate scripts

After the plots are finished they should be transfered (fast) to a temporary storage on the plotter (consumer SSD) by rsync or another way.
From this temp storage the farmer can pick it up over the network.
Note: If your network is fast enough (10G) you may copy the final files directly to the farmer. DO NOT use “-d” for NSF or remote storage (last stage compression).

farmer-gettheplots.sh

The farmer stores the plots on a temporary storage (hard drive) again. This is related to prevent a jam when transfering the files over the network in case of slow final drive speeds.

I implented this cause I have seen a drop on large hard drives when they are filling up the last 5% of free hard disk space.

farmer-movetheplots.sh

After the files are on the farmer, you can distribute them on the free (final) disks from the temp storages.

More Questions?

If there are still simple questions, feel free to ask them in the comments. Complicated issues can also be solved at reasonable hourly rates..

[Advertising] Finally, I’d like to thank Millionminer.de for supporting me monetarily in publishing the scripts for Chia.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

code