Introduction

Welcome to the LGO Node.

This open source tool allows you to browse the order batches created on LGO platform.

It uses data published by LGO and its Certification Authority to decrypt and validate the trading activity.

Browse batches

Documentation

Overview of the solution

The process used by the LGO platform is:

  • The Certificate Authority publishes public keys that have a validity period
  • The traders use these public keys to cipher their orders before sending them to LGO
  • LGO builds batches with received orders, publishes the hash of the batch in OpenTimestamp, and publishes the encrypted orders in a bucket
  • LGO requests the Certificate Authority to decipher the batch content, given that the OpenTimestamp proof is published
  • LGO executes the orders of the batch
  • LGO publishes the orders that were not executable and the reason of failure
  • When the public keys expire, their corresponding private keys are released by the CA

Batching

Batch hash calculation

Batch's hash is a SHA256 hash of the batch content plus last bast's hash for chaining.

The batch's hash contains, in this order:

  1. The hash of the previous batch (or the orders hash for the first batch of the chain)
  2. The orders hash

The orders hash is also a SHA256 hash of the orders list. It contains, for each order:

  1. The order identifier
  2. The order public key identifier
  3. The order encrypted data

Public data produced by the platform

For each batch, the LGO platform produces 5 files in a GCS bucket. These files are stored in a directory named with the identifier of the batch padded on ten zeros.

Ex: batch 642638 can be found at gs://lgo-markets-batches/0000642638.

These files are:

  • The batch metadata file 0000642638.metadata
  • The encrypted order list 0000642638.avro
  • The failed orders list 0000642638-failed-orders.avro
  • The OpenTimestamps proof and upgraded proof 0000642638.ots and 0000642638.ots.upgraded.ots

LGO platform configuration

Order checks

  • quantity : (in base currency for a limit order and a market sell order, in quote currency for a market buy order)

    min ≤ remaining quantity ≤ max

  • price (quote currency for limit order)

    min ≤ price ≤ max

  • amount (price × remaining quantity, quote currency)

    min ≤ amount ≤ max

  • price increment

    price is a multiple of quote increment

Market orders are only checked on quantity.

Actual limits are, for BTC/USD pair:

LimitValue
BTC minBTC 0.001
BTC maxBTC 1000
USD minUSD 10
USD maxUSD 1000000
USD price incrementUSD 0.10
Amount minUSD 10
Amount maxUSD 50000000

Algorithms

  • Actual Self Trade Prevention behavior: cancel remaining quantity of resting order
  • Actual Matching algorithm: FIFO

LGO orders processing

Orders reception and decryption

The general steps and their associated order invalidity reasons before batch execution are:

  1. Receive the order.
  2. Verify that the key identifier is not blank (INVALID_KEY_ID).
  3. Verify the order payload (INVALID_ORDER_PAYLOAD).
  4. Verify the order signature (INVALID_SIGNATURE).
  5. When the batch is ready, compute the hash of the batch content and timestamp the hash with OpenTimestamps.
  6. Publish the batch metadata, the encrypted orders and the OpenTimestamps proof in a public bucket.
  7. Request decryption from the CA.
  8. If public key was not valid, add INVALID_KEY to failed orders.
  9. Check the payload, if invalid : INVALID_PAYLOAD.
  10. When the timestamp transaction is validated on the Bitcoin blockchain, the OpenTimestamps proof is upgraded

Order execution

After the orders are decrypted and their content is checked, try to execute them, in order.

  1. Start batch execution

  2. Was the cancellation of all orders in order books requested ? (Information available as a boolean in the metadata file). If yes, all resting orders of all order books are canceled.

  3. Place all cancel orders

    • For each cancel order, try to cancel a corresponding resting order.
    • If order to cancel not from the same trader, add NOT_OWNER_OF_ORDER_TO_CANCEL in cancelled execution list (will be published in failed orders Avro file)
    • If order not found in books, save cancel order for in-batch cancellation.
  4. Place all orders

    • For each order, one by one (ordered execution), try to execute order.
      1. Verify if it should be canceled. If yes, verify if cancel order is from same trader, if not, add the cancel order and the reason NOT_OWNER_OF_ORDER_TO_CANCEL in cancelled execution list

      2. Pre-trade securities

        • Check product : order pair should be a valid LGO product (reason: INVALID_PRODUCT)

        • Check limits :

          • Market order: check quantity (INVALID_QUANTITY)
          • Limit order: check quantity (INVALID_QUANTITY), price (INVALID_PRICE), amount (INVALID_AMOUNT), price increment (INVALID_PRICE_INCREMENT)
        • Check funds : trader should own order amount plus taker fees amount (for BUY order) (INSUFFICIENT_FUNDS)

          Stop at the first check failing, add the order id and the failure reason to cancelled execution list

      3. Try to match order with right order book side

        • Loop:
          • Get top resting order
          • if cannot match, stop
          • if is a self trade match, execute STP algorithm and continue
          • create trade, cancel resting and taker orders if remaining too low, dequeue resting order if no remaining
          • if taker order has been entirely matched, stop
        • If order has a remaining quantity and is a limit order, enter order book.
  5. End batch

    • Publish cancelled executions list

Verify our fairness

To verify our fairness, you should, for each batch:

  1. Verify the OpenTimestamps proof
  2. Read the Avro content of orders and failed orders
  3. Verify the hash of the batch
  4. Verify that the hash of the previous batch is the same as your computation of the previous batch hash
  5. Get the private keys of the batch orders : gs://lgo-markets-keys/theOrderKeyId
  6. Decipher the orders data
  7. Parse the order content
  8. Execute your matching algorithm

Matching algorithm for the public data

After deciphering the orders and reading the failed orders details;

For each order in the same order as found in the Avro file:

  • verify if the order is present in the cancelled execution list (except with a Canceled by STP reason), if so, stop

  • loop

    • verify is the top order of the corresponding order book side is in the cancelled execution list with details containing counterOrderId:the id of the currently executed order. If so, remove top resting order from book (it was cancelled by STP)
    • match the order with the top order of the corresponding order book side, if the price matches (see note below)
    • if the top resting order is entirely matched or has a remaining quantity lower than limit, remove it from the book side
    • if the executed order is entirely matched or has a remaining quantity lower than limit, stop
    • if the price does not match the top resting order, break
  • if the order is a market order, stop

  • if the order is a limit order and has sufficient remaining quantity, add it to the right book side at the end of its price level

NB:

  • A market order always matches the top resting order.
  • For a sell limit order, order price should be ≤ to the resting order price.
  • For a buy limit order, order price should be ≥ to the resting order price.

When two orders match, the trade is made with the resting order price.

Detailed specifications

Example of batch hash calculation in JAVA:

import com.google.common.base.Charsets;
import com.google.common.hash.Hasher;
import com.google.common.hash.Hashing;

public String batchHash(List<EncryptedOrder> encryptedOrders, String lastBatchHash) {
    String ordersHash = ordersHash(encryptedOrders);
    return Hashing.sha256().newHasher()
            .putString(lastBatchHash == null ? ordersHash : lastBatchHash, Charsets.UTF_8)
            .putString(ordersHash, Charsets.UTF_8)
            .hash().toString();
}

private String ordersHash(List<EncryptedOrder> encryptedOrders) {
    Hasher hasher = Hashing.sha256().newHasher();
    for (EncryptedOrder encryptedOrder : encryptedOrders) {
        String hash = Hashing.sha256().newHasher()
                .putLong(encryptedOrder.id)
                .putString(encryptedOrder.keyId, Charsets.UTF_8)
                .putBytes(encryptedOrder.data).hash().toString();
        hasher.putString(hash, Charsets.UTF_8);
    }
    return hasher.hash().toString();
}

Public data matching algorithm

Order books: All order books. One order book by product (currency pair), each book has two sides (buy and sell).

Buy side: Treemap of long (price) and Linked list (resting orders FIFO), keys ordered in opposite order

Sell side: Treemap of long (price) and Linked list (resting orders FIFO), keys ordered in natural order

Orders: List or deciphered orders

Failed orders: Map of long (order id) and failed orders details

Cancel orders: A list of order id to cancel, built from the cancel order requests that did not fail

if metadata.cancelOrderBooks then empty all order books sides

for each cancel order, apply on orders books

for each takerOrder in orders
    if takerOrder could not be executed (1)
        continue
    if takerOrder was canceled
        continue
    book ← orderBooks[takerOrder.base, takerOrder.quote]
    orderBookSide ← takerOrder.direction is BUY ? book.sellSide : book.buySide
    while takerOrder has sufficient remaining
        restingOrder ← orderBookSide.first.head
        if restingOrder was canceled by STP by takerOrder (2)
            remove restingOrder from orderBookSide
            continue
        if price does not match between takerOrder and restingOrder (5)
            break
        consume takerOrder and restingOrder to the max possible quantity
        if restingOrder has not sufficient remaining (3)
            remove restingOrder from orderBookSide
    if takerOrder has sufficient remaining (3) and is a limit order
        add takerOrder to the order book (buy side for buy order, sell side for sell order)

(1) takerOrder could not be executed :

failedOrders contains takerOrderId 
    and failedOrder[takerOrderId].reason ≠ "OrderCanceledBySTP"

(2) restingOrder was canceled by STP by takerOrder :

failedOrders contains restingOrderId 
    and failedOrders[restingOrderId].reason = "OrderCanceledBySTP" 
    and failedOrders[restingOrderId].details["counterOrderId"]="takerOrderId"

(3) order has sufficient remaining :

order.remainingQuantity > 0 and limits are valid (4)

(4) limits are valid :

  • for a market order buy:

      USD min ≤ remaining quantity ≤ USD max
    
  • for a market order sell:

      BTC min ≤ remaining quantity ≤ BTC max
    
  • for a limit order :

      BTC min ≤ remaining quantity ≤ BTC max
      USD min ≤ price ≤ USD max
      USD amount min ≤ remaining quantity × price ≤ USD amount max
      price % USD price increment = 0
    

(5) price matches between an order and a resting order

  • for a market order

      true
    
  • for a sell limit order

      order.price ≤ restingOrder.price
    
  • for a buy limit order

      order.price ≥ restingOrder.price
    

FAQ

Why is the transparency tool not exactly up to date with the published batches?

First, in order to decipher the orders without asking for the CA intervention, the private keys need to be released by the CA, which can not happen during the validity period of the public key.

Secondly, LGO node waits for the validation of the timestamp transaction on the Bitcoin blockchain. Depending on Bitcoin mempool state, it can take between 10 minutes up to 10 hours.

What is the tLGO1/tLGO2 product?

The tLGO1/tLGO2 product is a test product only used by the technical team to check that the platform is up and the trading is possible.