#5 - BUIDL an Auto-compounding Bot on Saber

Previous#4 - BUIDL a Swap UI on Solana Next#6 - A Starter Kit for Running Solana Validator

Last updated 1 year ago

Authors: @wei_sol_, @ironaddicteddog

[Updated at 2022.3.31]

See the example repo here

TL; DR

Build SDKs only using @solana/web3.js
Learn how to interact with Solana
Builld an auto-compounding bot with SDK

Introduction

Solana : Solana is a fast, low cost, decentralized blockchain with thousands of projects spanning DeFi, NFTs, Web3 and more.

Saber : Saber is a Curve-like AMM provider on Solana, support all kinds of stablecoin pair from various bridges

Solana 101

Account : Everything on Solana is an account * Accounts can only be owned by programs * Every Account is like a file in a computer * Accounts are used to store state * Only the account owner may debit an account and adjust its data * All accounts to be written to or read must be passed into Insructions * Developers should use the data field to save data inside accounts >Image from https://paulx.dev/blog/2021/01/14/programming-on-solana-an-introduction/

Image from https://explorer.solana.com/address/6ZRCB7AAqGre6c72PRz3MHLC73VMYvJ8bi9KHf1HFpNk

Program : Program is just an account with excutable enable * Solana programs are stateless * Designed be upgradable (BPF loader 2)

Image from https://explorer.solana.com/address/TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA

Program Account : An Account owned by a Program other than System program * Data struct are very different from program to program

Image from https://explorer.solana.com/address/EzkjmZFzWccf2DRQ6uafahvfZh29ntwAmej8nDQ5tY1C

Keyair : A byte array that contain Public key and Private key * first 32 bytes are use as Private key * last 32 bytes are generated by Private key and use as Public key

[26,151,18,191,115,212,220,144,52,66,74,133,251,235,69,161,254,121,70,227,171,227,17,170,154,227,32,151,40,125,37,158,0,94,253,210,209,242,208,122,162,84,158,36,211,63,243,252,104,36,58,243,120,134,127,132,193,186,63,50,0,230,93,200] 12T1dsupQBqwgQYsXWqhhmzQgreRtvkP95W8rW3Pk23R

Address : A Public key encoded in Base58

Rent : Pay for space to store data on Solana * If Rent is paid for over 2 years, the account become rent-exempt

Image from https://explorer.solana.com/address/8UviNr47S8eL6J3WfDxMRa3hvLta1VDJwNWqsDgtN3Cv

RPC : The endpoint to read/write from/to Solana * Free RPCs can be slow or congested when there is high demand.

PDA (Program Derived Address) : A account without a Private key that only can be signed by a Program * PDA is generated by hashing a seed with the Program address

Instruction : The one and only way to interact with Programs * All accounts used when processing should be write into the Instruction

Raw instruction from https://explorer.solana.com/tx/v2s26c5vBzrc7rTEyEsCreBDHaVd41iJ8F4j1gLFAy1QA5q8tqJex2Y2h3xsDmJpU1R9wAyRqziyzNB85cgxFjh

Serialized instruction https://explorer.solana.com/tx/37oPunF5tLw6DEqQUk4gMHeb6n1wdCAPWBF7e291znWwPXW7gCvPh9Srv5m5UMoxZoHSMR4BcQau1RnmU5yQeWKj

Tx (Transaction) : A message send to RPC that contains one or more Instructions, Signatures and a Fee Payer * Every Tx have a size limit of 1232 bytes * A Tx can be signed by differents accounts * Fee are determined by the amount of Signer at the moment

Image from https://explorer.solana.com/tx/v2s26c5vBzrc7rTEyEsCreBDHaVd41iJ8F4j1gLFAy1QA5q8tqJex2Y2h3xsDmJpU1R9wAyRqziyzNB85cgxFjh

SPL (Solana Program Library) : A example library organized by Solana lab * SPL is not a token protocol on Solana, spl-token is.

ATA (Associated Token Account) : A Token Account which is a PDA created by Associated Token Program * There is only one ATA with every wallet and a token mint

Image from 白上フブキ.eth

Solana system model

Overview

Learn Solana basic from building a SDK
Build a bot that collect, sell, reinvest the yield from LP farming

Architecture

File Structure

├── 📂 raydium
│   │
│   ├── 📄 ids.ts
│   │
│   ├── 📄 index.ts
│   │
│   ├── 📄 infos.ts
│   │
│   ├── 📄 instructions.ts
│   │
│   ├── 📄 layouts.ts
│   │
│   └── 📄 transactions.ts
│
│
├── 📂 saber
│   │
│   ├── 📄 ids.ts
│   │
│   ├── 📄 index.ts
│   │
│   ├── 📄 infos.ts
│   │
│   ├── 📄 instructions.ts
│   │
│   ├── 📄 layouts.ts
│   │
│   └── 📄 transactions.ts
│
│
│── 📄 index.ts
│
│── 📄 utils.ts
│
│── 📄 package.json
│
│── 📄 tsconfig.json
│
└── ...

Setup

Install Rust and Solana

$ sh -c "$(curl -sSfL https://release.solana.com/v1.9.8/install)"
...

See https://hackmd.io/@ironaddicteddog/solana-starter-kit#Install-Rust-and-Solana-Cli for more details.

Recover your Wallet (Using Phantom)

FIrst, click Show Secret Reconvery Phrase and copy your recovery phrase at this point.

Next, let's recover the wallet locally:

$ solana-keygen recover 'prompt:?key=0/0' -o ~/.config/solana/solmeet-keypair-1.json

There should be a prompt asking for entering the recovery phrase in yout terminal. Paste your recovery phrase at this point.

Set keypair

$ solana config set --keypair ~/.config/solana/solmeet-keypair-1.json

Config to `solana-mf`

$ solana config set --url https://rpc-mainnet-fork.dappio.xyz
$ solana config set --ws wss://rpc-mainnet-fork.dappio.xyz/ws
$ solana config set --commitment processed
$ solana airdrop 1

Scaffold

$ mkdir solmeet-5-bot
$ cd solmeet-5-bot
$ tsc --init

$ touch {index.ts,utils.ts}
$ mkdir saber && touch saber/{index.ts,ids.ts,layouts.ts,infos.ts,instructions.ts,transactions.ts}
$ mkdir raydium && touch raydium/{index.ts,ids.ts,layouts.ts,infos.ts,instructions.ts,transactions.ts}

Add `package.json`

{
  "name": "solmeet-5-bot",
  "version": "1.0.0",
  "description": "",
  "main": "./index.ts",
  "scripts": {
    "start": "ts-node ./index.ts"
  },
  "dependencies": {
    "@project-serum/borsh": "^0.2.5",
    "@project-serum/serum": "^0.13.61",
    "@solana/buffer-layout": "^4.0.0",
    "@solana/spl-token": "^0.2.0",
    "@solana/web3.js": "^1.35.0",
    "bignumber.js": "^9.0.1",
    "buffer-layout": "^1.2.2",
    "js-sha256": "^0.9.0"
  },
  "devDependencies": {
    "@project-serum/borsh": "^0.2.5",
    "@solana/web3.js": "^1.35.0",
    "@types/express": "^4.17.13",
    "@types/node": "^17.0.18",
    "buffer-layout": "^1.2.2",
    "ts-node": "^10.5.0",
    "typescript": "^4.5.5"
  }
}

Install Dependencies

$ yarn

Part 1: Implement Common Modules

In this part, we will implement the common modules for the bot:

ids.ts
layouts.ts
utils.ts

`ids.ts`

In Solana, execution (programs) and states are decoupled. As a result, we have to be very clear on the scope of the the programs and states:

`saber/ids.ts`

import { PublicKey } from "@solana/web3.js";

export const SBR_MINT = new PublicKey("Saber2gLauYim4Mvftnrasomsv6NvAuncvMEZwcLpD1");
export const USDC_UST_POOL = new PublicKey("KwnjUuZhTMTSGAaavkLEmSyfobY16JNH4poL9oeeEvE");
export const ADMIN_KEY = new PublicKey("H9XuKqszWYirDmXDQ12TZXGtxqUYYn4oi7FKzAm7RHGc");
export const SWAP_PROGRAM_ID = new PublicKey("SSwpkEEcbUqx4vtoEByFjSkhKdCT862DNVb52nZg1UZ");
export const SABER_WRAP_PROGRAM_ID = new PublicKey("DecZY86MU5Gj7kppfUCEmd4LbXXuyZH1yHaP2NTqdiZB");
export const SABER_QUARRY_REWARDER = new PublicKey("rXhAofQCT7NN9TUqigyEAUzV1uLL4boeD8CRkNBSkYk");
export const QURARRY_MINE_PROGRAM_ID = new PublicKey("QMNeHCGYnLVDn1icRAfQZpjPLBNkfGbSKRB83G5d8KB");
export const SABER_MINT_WRAPPER = new PublicKey("EVVDA3ZiAjTizemLGXNUN3gb6cffQFEYkFjFZokPmUPz");
export const QURARRY_MINT_WRAPPER = new PublicKey("QMWoBmAyJLAsA1Lh9ugMTw2gciTihncciphzdNzdZYV");
export const SABER_FARM_MINTER = new PublicKey("GEoTC3gN12qHDniaDD7Zxvd5xtcZyEKkTPy42B44s82y");
export const IOU_TOKEN_MINT = new PublicKey("iouQcQBAiEXe6cKLS85zmZxUqaCqBdeHFpqKoSz615u");
export const CLAIM_FEE_TOKEN_ACCOUNT = new PublicKey("4Snkea6wv3K6qzDTdyJiF2VTiLPmCoyHJCzAdkdTStBK");
export const SABER_TOKEN_MINT = new PublicKey("Saber2gLauYim4Mvftnrasomsv6NvAuncvMEZwcLpD1");
export const MINTER_PROGRAM_ID = new PublicKey("RDM23yr8pr1kEAmhnFpaabPny6C9UVcEcok3Py5v86X");
export const DEPRECATED_POOLS = [
  new PublicKey("LeekqF2NMKiFNtYD6qXJHZaHx4hUdj4UiPu4t8sz7uK"),
  new PublicKey("2jQoGQRixdcfuRPt9Zui7pk6ivnrQv79mf8h13Tyoa9K"),
  new PublicKey("SPaiZAYyJBQHaSjtxFBKtLtQiCuG328r1mTfmvvydR5"),
  new PublicKey("HoNG9Z4jsA1qtkZhDRYBc67LF2cbusZahjyxXtXdKZgR"),
  new PublicKey("4Fss9Dy3vAUBuQ4SyEZz4vcLxeQqoFLZjdXhEUr3wqz3")
]

`raydium/ids.ts`

import { PublicKey } from "@solana/web3.js";

export const SBR_AMM_ID = new PublicKey("5cmAS6Mj4pG2Vp9hhyu3kpK9yvC7P6ejh9HiobpTE6Jc")
export const LIQUIDITY_POOL_PROGRAM_ID_V3 = new PublicKey('27haf8L6oxUeXrHrgEgsexjSY5hbVUWEmvv9Nyxg8vQv')
export const LIQUIDITY_POOL_PROGRAM_ID_V4 = new PublicKey('675kPX9MHTjS2zt1qfr1NYHuzeLXfQM9H24wFSUt1Mp8')
export const STAKE_PROGRAM_ID = new PublicKey('EhhTKczWMGQt46ynNeRX1WfeagwwJd7ufHvCDjRxjo5Q')
export const STAKE_PROGRAM_ID_V5 = new PublicKey('9KEPoZmtHUrBbhWN1v1KWLMkkvwY6WLtAVUCPRtRjP4z')
export const AMM_AUTHORITY = new PublicKey("5Q544fKrFoe6tsEbD7S8EmxGTJYAKtTVhAW5Q5pge4j1")

`layouts.ts`

layouts play an important role in both reading and writing on-chain data.

Reading: It indicates how the stored bytes is arranged and what their types are.
Writing: It indicates how the instruction data should be assembled to call a certain program.

`saber/layouts.ts`

import { publicKey, struct, u64, u128, u8, u16, i64, bool } from "@project-serum/borsh";

export const FARM_LAYOUT = struct([
  publicKey("rewarderKey"),
  publicKey("tokenMintKey"),
  u8("bump"),
  u16("index"),
  u8("tokenMintDecimals"),
  i64("famineTs"),
  i64("lastUpdateTs"),
  u128("rewardsPerTokenStored"),
  u64("annualRewardsRate"),
  u64("rewardsShare"),
  u64("totalTokensDeposited"),
  u64("numMiners"),
]);

export const MINER_LAYOUT = struct([
  publicKey("farmKey"),
  publicKey("owner"),
  u8("bump"),
  publicKey("vault"),
  u64("rewardsEarned"),
  u128("rewardsPerTokenPaid"),
  u64("balance"),
  u64("index"),
]);

export const SWAPINFO_LAYOUT = struct([
  bool("isInitialized"),
  bool("isPaused"),
  u8("nonce"),
  u64("initialAmpFactor"),
  u64("targetAmpFactor"),
  i64("startRampTs"),
  i64("stopRampTs"),
  i64("futureAdminDeadline"),
  publicKey("futureAdminKey"),
  publicKey("adminKey"),
  publicKey("tokenAccountA"),
  publicKey("tokenAccountB"),
  publicKey("poolMint"),
  publicKey("mintA"),
  publicKey("mintB"),
  publicKey("adminFeeAccountA"),
  publicKey("adminFeeAccountB"),
]);

export const WRAPINFO_LAYOUT = struct([
  u8("decimal"),
  u64("multiplyer"),
  publicKey("underlyingWrappedTokenMint"),
  publicKey("underlyingTokenAccount"),
  publicKey("wrappedTokenMint"),
]);

export const DEPOSIT_LAYPOUT = struct([
  u8('instruction'),
  u64('AtokenAmount'),
  u64('BtokenAmount'),
  u64('minimalRecieve'),
]);

export const WITHDRAW_LAYOUT = struct([
  u8('instruction'),
  u64('LPtokenAmount'),
  u64('minimalRecieve'),
]);

export const WRAP_LAYOUT = struct([
  u64('amount'),
]);

export const UNWRAP_LAYOUT = struct([
  u64('amount'),
]);

export const DEPOSIT_TO_FARM_LAYOUT = struct([
  u64('amount'),
]);

export const CREATE_MINER_LAYOUT = struct([
  u64('amount'),
]);

export const WITHDRAW_FROM_FARM_LAYOUT = struct([
  u64('amount'),
]);

`raydium/layouts.ts`

import { publicKey, struct, u8, u64, u128 } from "@project-serum/borsh";

export const SWAP_LAYOUT = struct([
  u8('instruction'),
  u64('amountIn'),
  u64('minAmountOut')
]);

export const ADD_LIQUIDITY_LAYOUT = struct([
  u8('instruction'),
  u64('maxCoinAmount'),
  u64('maxPcAmount'),
  u64('fixedFromCoin')
]);

export const REMOVE_LIQUIDITY_LAYOUT = struct([
  u8('instruction'),
  u64('amount')
]);

export const AMM_INFO_LAYOUT_V4 = struct([
  u64("status"),
  u64("nonce"),
  u64("orderNum"),
  u64("depth"),
  u64("coinDecimals"),
  u64("pcDecimals"),
  u64("state"),
  u64("resetFlag"),
  u64("minSize"),
  u64("volMaxCutRatio"),
  u64("amountWaveRatio"),
  u64("coinLotSize"),
  u64("pcLotSize"),
  u64("minPriceMultiplier"),
  u64("maxPriceMultiplier"),
  u64("systemDecimalsValue"),
  // Fees
  u64("minSeparateNumerator"),
  u64("minSeparateDenominator"),
  u64("tradeFeeNumerator"),
  u64("tradeFeeDenominator"),
  u64("pnlNumerator"),
  u64("pnlDenominator"),
  u64("swapFeeNumerator"),
  u64("swapFeeDenominator"),
  // OutPutData
  u64("needTakePnlCoin"),
  u64("needTakePnlPc"),
  u64("totalPnlPc"),
  u64("totalPnlCoin"),
  u128("poolTotalDepositPc"),
  u128("poolTotalDepositCoin"),
  u128("swapCoinInAmount"),
  u128("swapPcOutAmount"),
  u64("swapCoin2PcFee"),
  u128("swapPcInAmount"),
  u128("swapCoinOutAmount"),
  u64("swapPc2CoinFee"),
  publicKey("poolCoinTokenAccount"),
  publicKey("poolPcTokenAccount"),
  publicKey("coinMintAddress"),
  publicKey("pcMintAddress"),
  publicKey("lpMintAddress"),
  publicKey("ammOpenOrders"),
  publicKey("serumMarket"),
  publicKey("serumProgramId"),
  publicKey("ammTargetOrders"),
  publicKey("poolWithdrawQueue"),
  publicKey("poolTempLpTokenAccount"),
  publicKey("ammOwner"),
  publicKey("pnlOwner"),
]);

`utils.ts`

Copy this code snippet to utils.ts.

`getAnchorInsByIdl`

Calculate Anchor Identifier:

// Example

function getAnchorInsByIdl(name: string): Buffer {
  const SIGHASH_GLOBAL_NAMESPACE = "global";
  const preimage = `${SIGHASH_GLOBAL_NAMESPACE}:${name}`;
  const hash = sha256.sha256.digest(preimage)
  const data = Buffer.from(hash).slice(0, 8)
  return data;
}

Part 2: Implement "Read" Modules

DataSizeFilter and MemCmpFilter

// Example

const adminIdMemcmp: MemcmpFilter = {
  memcmp: {
    offset: 8,
    bytes: rewarderKey.toString(),
  }
};

const sizeFilter: DataSizeFilter = {
  dataSize: 140
}
const filters = [adminIdMemcmp, sizeFilter];
const config: GetProgramAccountsConfig = { filters };
const allFarmAccount = await connection.getProgramAccounts(QURARRY_MINE_PROGRAM_ID, config);

`infos.tx`

Copy this code snippet to saber/infos.ts and this code snippet to raydium/infos.ts.

`index.ts`

Copy this code snippet to saber/index.ts and this code snippet to raydium/index.ts.

Part 3: Implement "Write" Modules

Need to add Anchor identifier manually
One single Solana tx includes multiple ixs
- Solana ix = Ethereum tx
- Solana tx = Ethereum multicall

`instructions.ts`

Copy this code snippet to saber/instructions.ts and this code snippet to raydium/instructions.ts.

`transactions.ts`

Copy this code snippet to saber/transactions.ts and this code snippet to raydium/transactions.ts.

Part 4: Implement the Auto-compounding Bot

Load Keypair
Claim All mining rewards
Swap all SBR to USDC
Add all USDC swapped out to USDC-UST pool
Deposit all LP to farming

`index.ts`

import os from "os";
import fs from "fs";
import BN from "bn.js";
import { Connection, Keypair } from "@solana/web3.js";
import * as raydium from "./raydium";
import { SBR_AMM_ID } from "./raydium/ids";
import * as saber from "./saber";
import { SBR_MINT, USDC_UST_POOL } from "./saber/ids";
import * as utils from "./utils";

// Load keypair
const keyPairPath = `${os.homedir()}/.config/solana/solmeet-keypair-1.json`;
const privateKeyUint8Array = JSON.parse(fs.readFileSync(keyPairPath, "utf-8"));
const privateKey = Uint8Array.from(privateKeyUint8Array);
const wallet = Keypair.fromSecretKey(privateKey);

async function main() {
  const conn = new Connection("https://rpc-mainnet-fork.dappio.xyz", { wsEndpoint: "wss://rpc-mainnet-fork.dappio.xyz/ws", commitment: "processed", });
  // const connection = new Connection("https://solana-api.tt-prod.net", { commitment: "processed", });
  console.log("Fetching all Saber pools...");
  const swaps = await saber.getAllSwaps(conn);
  console.log("Fetching all Saber miners...");
  const miners = await saber.getAllMiners(conn, wallet.publicKey);
  console.log("Fetching Saber AMM pool on Raydium...");
  const sbrAmm = (await raydium.getAmmPool(SBR_AMM_ID, conn));

  // Claim All mining rewards
  console.log("Claiming all mining rewards...")
  for (const miner of miners) {
    for (const swap of swaps) {
      if (miner.farmKey.toString() === swap.farmingInfo?.infoPubkey.toString()) {
        if (miner.balance.toNumber() > 0) {
          // Create claimRewardTx
          const claimRewardTx = await saber.claimRewardTx(swap.farmingInfo as saber.FarmInfo, wallet.publicKey, conn)
          // Send Tx
          const result = await utils.signAndSendAll(claimRewardTx, conn, wallet)
          console.log(miner.getUnclaimedRewards(swap), "SBR reward claimed. Tx:", result);
        }
      }
    }
  }

  let tokenAccounts = await utils.getAllTokenAccount(wallet.publicKey, conn);
  let swapOutAmount = new BN(0);

  // Swap all SBR to USDC
  console.log("Swapping all SBR to USDC...")
  for (const token of tokenAccounts) {
    if (token.mint === SBR_MINT && token.amount.cmpn(0)) {
      swapOutAmount = await (await sbrAmm.calculateSwapOutAmount("coin", token.amount, conn)).divn(0.98);
      if (!swapOutAmount.cmpn(1)) {
        break;
      }
      const swapIx = await raydium.swap(sbrAmm, token.mint, sbrAmm.pcMintAddress, wallet.publicKey, token.amount, new BN(0), conn);
      const result = await utils.signAndSendAll(swapIx, conn, wallet);
      console.log(token.amount.toNumber() / 1000000, "SBR swapped. Tx:", result);
    }
  }

  // Add all USDC swapped out to USDC-UST pool
  console.log("Adding all USDC swapped out to USDC-UST pool...")
  for (const swap of swaps) {
    if (swap.infoPublicKey === USDC_UST_POOL) {
      const addLP = await saber.createDepositTx(swap, new BN(0), swapOutAmount, new BN(0), wallet.publicKey, conn)
      const result = await utils.signAndSendAll(addLP, conn, wallet)
      console.log("LP reinvested. Tx:", result);
    }
  }

  // Deposit all LP to farming
  console.log("Depositing all LP to farming...")
  tokenAccounts = await utils.getAllTokenAccount(wallet.publicKey, conn)
  for (const swap of swaps) {
    for (const token of tokenAccounts) {
      if (token.mint.toString() === swap.poolMint.toString() && token.amount.cmpn(0)) {
        // Create farmIx
        const farmIx = await saber.depositToFarm(swap.farmingInfo as saber.FarmInfo, wallet.publicKey, token.amount, conn)
        // Send Tx
        const result = await utils.signAndSendAll(farmIx, conn, wallet)
        console.log("Farm deposited. Tx:", result);
      }
    }
  }
}

async function run() {
  try {
    main();
  }
  catch (e) {
    console.error(e);
  }
}

run();

Finally, let's run the bot:

$ yarn start
Fetching all Saber pools...
Fetching all Saber miners...
Fetching Saber AMM pool on Raydium...
Claiming all mining rewards...
0.008659 SBR reward claimed. Tx: 5xCibLpPokio4YHTwVZ35VM8fG8cww8Ris52E1D1qr2F8VSgdJsrHoBJGRDE77p61kXU3UwTMYKtEP3VB2fj1tFM
0 SBR reward claimed. Tx: 2mwEVHKwQwG384JM9ys9iAoKjRnZVswyQRC4hip44nHPP2uJvkA1YUwLfrDn9bk84GxUpPyZjpjzkxZ2ENmfQPb4
Swapping all SBR to USDC...
Adding all USDC swapped out to USDC-UST pool...
Depositing all LP to farming...
✨  Done in 121.22s.

References

https://github.com/DappioWonderland/auto-compounding-bot
https://hackmd.io/@ironaddicteddog/solana-starter-kit
https://hackmd.io/@ironaddicteddog/solana-anchor-escrow

TL; DR

Introduction

Solana 101

Solana system model

Overview

Architecture

File Structure

Setup

Install Rust and Solana

Recover your Wallet (Using Phantom)

Config to solana-mf

Scaffold

Add package.json

Install Dependencies

Part 1: Implement Common Modules

ids.ts

saber/ids.ts

raydium/ids.ts

layouts.ts

saber/layouts.ts

raydium/layouts.ts

utils.ts

getAnchorInsByIdl

Part 2: Implement "Read" Modules

infos.tx

index.ts

Part 3: Implement "Write" Modules

instructions.ts

transactions.ts

Part 4: Implement the Auto-compounding Bot

index.ts

References

Config to `solana-mf`

Add `package.json`

`ids.ts`

`saber/ids.ts`

`raydium/ids.ts`

`layouts.ts`

`saber/layouts.ts`

`raydium/layouts.ts`

`utils.ts`

`getAnchorInsByIdl`

`infos.tx`

`index.ts`

`instructions.ts`

`transactions.ts`

`index.ts`