Scanners

As explained in the Concepts section, scanners filter stocks of interest from the stock universe. When the trader application starts, it reads the tradeplan.toml file and looks at the [[scanners]] section to identify the classes of scanners to be instantiated and executed. They are executed by the order of appearance in the tradeplan file. The framework comes with a generic, momentum scanner, but it is advised to be used as a reference for designing your own scanners.

Some configuration parameters are generic to all scanners, and some may be specific for your own scanner.

No Liability Disclaimer

Any example, or sample is provided for training purposes only. You may choose to use it, modify it or completely disregard it - LiuAlgoTrader and its authors bare no responsibility to any possible loses from using a scanner, strategy, miner, or any other part of LiuAlgoTrader (on the other hand, they also won’t share your profits, if there are any).

Developing a Scanner

When developing your scanner, you first need to import base-class Scanner:

import alpaca_trade_api as tradeapi
from liualgotrader.scanners.base import Scanner

Creating a scanner involves:

1. Declare a class object that is inherited from base-class Scanner,
2. Overwrite both the __init__() and run() methods. __init__() is called during initialization, and is passed the configuration parameters from the tradeplan.toml file. The run() is called when the framework is ready to execute the scanner.

The __init__() function

A Generic implementation for the __init__() function:

def __init__(
    self,
    name: str,
    data_loader: DataLoader,
    recurrence: Optional[timedelta],
    target_strategy_name: Optional[str],
    data_source: object = None,
)

The run() function

The run() method is at the heart of the Scanner. The skeleton implementation:

async def run(self, back_time: datetime = None) -> List[str]:

The run() function returns a list of stock symbols which the framework needs to track. back_time is used by the backtester to simulate past time-stamps, allowing developer to test various scenarios before deployment.

self.data_loader

Holds the generic interface for querying the stock universe.

Using DataLoader() class

The DataLoader() class is core for writing scanners (as well as strategies). The class wraps on top of pandas.DataFrame() class which supports automatically downloading data.

For example, to load recent ohlc data for Apple Inc all you can do:

apple_ohlc = self.data_loader['AAPL'][-1]

Notes:

1. Data will be loaded based on the time-scale (min/hour/day) configured by the LiuAlgoTrader framework. During trading you can expect per-minute time-scale, during back-testing you can select your desired time-frame,
2. Data is loaded from the data-providers configured in the DATA_CONNECTOR environment variable, the framework canonizes column names and guarantees to include at least open-high-low-close data, as well as volume. Different data providers may include additional details (such as VWAP, average, count and more),
3. Calling self.data_loader.symbol_data grants direct access to the DataFrame class,
4. Calling self.data_loader.data_api grants access to the instance of the DataAPI class, that provides an abstraction for the different data-providers. Using this class, the scanner can get a list of available / tradable symbols. For more details, check the implementation of the momenutum scanner.

Behind the scenes

As a developer, I hate ‘magic’ that happens without my understanding, hence it’s important for me to detail the inner workings of the framework. All scanners run inside a dedicated process. When the process is executed, it reads the configuration files specific to the scanners, and creates an asyncio task per scanner. In fact, it’s up to the scanners to make sure they play along nicely and not starve each other with overly long calculations.

The scanner task is to execute the run() method to determine a list of filtered stocks and transmits them over a Queue to the producer process. The task would then sleep() for the duration of the recurrence parameter (or just run once, if that parameter is not present).

Once the producer receives a list of selected symbols, it will register them for events on the Polygon.io data-stream, and persist them in database with the timestamp of receiving the selected stocks. This data are then used by the backtester application to replicate the real conditions presented to a strategy.

Example

An example of my_scanner.py:

"""my_scanner.py: custom scanner implementing the Scanner class"""
from datetime import datetime
from typing import List

from liualgotrader.common.data_loader import DataLoader  # type: ignore
from liualgotrader.scanners.base import Scanner


class MyScanner(Scanner):
    def __init__(self, data_loader: DataLoader, **kwargs):
        super().__init__(
            name=type(self).__name__,
            recurrence=kwargs.get("recurrence"),
            data_loader=data_loader,
            target_strategy_name=kwargs.get("target_strategy_name"),
        )

    async def run(self, back_time: datetime = None) -> List[str]:
        return ["AAPL"]

Configuring a custom scanner in the tradeplan TOML file is as easy as:

[scanners]
    [scanners.MyScanner]
        filename = "my_scanner.py"

        my_arg1 = 30000
        my_arg2 = 3.5

Upon execution, trader application will look for my_scanner.py and instantiate the MyScaner class, and then call it with the arguments defined in the tradeplan configuration file, while adding the trade-api object.

Advanced

Scanners can “direct” symbol picks to a specific strategy. To direct symbol picks, include target_strategy_name in the configuration file. That parameter, if present, will be passed along from the scanner process to the producer process, and later to the consumer process to make sure only the relevant strategy receives the pick.

Normally, the producer subscribes to all Polygon.io available events per picked stock. However, to improve performance, if the strategies do not make use of per-second events, or quotes or trades, it’s recommended to select only the relevant events in the configuration file:

events = ["second", "minute", "trade", "quote"]

Lastly, if configuration parameter test_scanners is set to true. The trader application will only execute the scanners without running strategies for debugging purpose.

Momentum Scanner

Scanners are defined in the tradeplan.toml TOML configuration file under the section [[scanners]]. Note that several Scanners may run, at different schedules and identify new stocks that needs to be fed into the Strategies pipeline.

The [scanners.momentum] momentum scanner has several properties:

Name	Description
provider	stock data provider: polygon, finnhub
min_volume	scan for tickers with minimal volume since day start
min_gap	tickers gaping up in percentage (e.g. 3.5)
min_last_dv	minimum last day dollar x volume
min_share_price	minimum stock price
max_share_price	maximum stock price
from_market_open	number of minutes to wait, since market open before scanning for relevant stocks
max_symbols	maximum number of symbols to scan for
recurrence	frequency of re-running scanner, if not present, run only once

Notes

• Unless specified, the trader applications scans & trades a limited number of stocks, however that limitation can be overwritten by LIU_MAX_SYMBOLS env variable.