US Equity Buy/Sell Pressure and Retail Indicators Minute Bar Guide

Version 1.0.0 (Feb 2025)

CONTACT US

We are here to help you do great things with our market and reference data. For questions, feedback, and other concerns, you may reach our team of experts using the following contact information:

algoseek customer support

support@algoseek.com

(+1) 646 583 1832

algoseek sales

sales@algoseek.com

(+1) 646 583 1832

TABLE OF CONTENTS

REVISION SUMMARY        4

INTRODUCTION        5

DATA ORGANIZATION AND FILE FORMAT        5

REVISION SUMMARY

INTRODUCTION

Identifying retail trades in Trade and Quote (TAQ) pricing data, specifically using the SIP (Securities Information Processor) feed, can be challenging because the SIP feed aggregates and normalizes data from multiple exchanges and doesn’t directly label whether a trade is retail or institutional. However, some indicators can be used to identify whether a trade is retail with high probability. For example, trade size, trade price, or the exchange where the trade happened can provide some information to identify retail trades.

The US Equity Buy/Sell Pressure and Retail Indicators Minute Bar dataset is built from top-of-book intraday quotes and trades for all listed stocks, ETNs, ETFs, ADRs, and funds from 15+ US exchanges and marketplaces. It provides a list of retail indicators and other analytics that can be helpful in such an analysis. The main approach to identifying retail trades is to analyze sub-penny pricing for FINRA/TRF trade reports (Tracking Retail Investor Activity).

DATA SOURCE

algoseek Buy/Sell Pressure and Retail Indicators Minute Bar dataset is built from “as-is” tick data collected from live SIP feed algoseek’s co-located ticker plant servers in Equinix NY2 and NY4 data centers, connected with 10Gb fiber for low latency.

The Securities Information Processor (SIP) includes Tape A and Tape B covered by the Consolidated Tape Association (CTA) plan and Tape C covered by the Unlisted Trading Privileges (UTP) plan. The SIP links the US markets by processing and consolidating all protected bid/ask quotes and trades from every trading venue into a single and easily consumable data feed.

The SIP disseminates and calculates critical regulatory information, including the National Best Bid and Offer (NBBO) and Limit Up Limit Down (LULD) price bands, among other important regulatory information such as short sale restrictions and regulatory halts. In the highly fragmented world of US equities, the SIP is an easy way for people to get a view of the current state of the market.

FINRA TRF AND ODD LOT TRADES

FINRA TRF

Equity trades are executed on Public Exchanges (e.g., NASDAQ, BATS, NYSE, ARCA, etc.) and off the public exchanges in Dark Pools, Broker-Dealer internal crossing, and Block Trades.

Regulation National Market System (NMS) requires all trades to be reported. There are currently three FINRA Trade Reporting Facilities (TRF) affiliated with registered national securities exchanges and provide FINRA members with a mechanism for reporting transactions affected otherwise than on an exchange.

Regulation NMS allows up to 10 seconds after the Trade execution time for the trade report to be sent to an exchange’s TRF for publication. The delay can result in TRF Trade reports printed on the market data feed being out of the current NBBO.

Round Lot and Odd Lot

A round lot (or board lot) is a normal unit of trading of a security, which currently is 100 shares of stock in the US. Any quantity less than 100 shares is referred to as an odd lot. Odd lots are not subject to the Regulation NMS rules requiring execution to be within the current NBBO. Broker-dealers send odd lots to the exchange paying the most rebate per share and not the best execution price.  Odd lot executions can create unrealistic high/low trade prices in an OHLC bar.

MINUTE BAR CALCULATIONS

Continuous Bar Time

algoseek Buy/Sell Pressure and Retail Indicators dataset provides continuous bars from pre-market opening (4 am ET), regular market hours, and post-market until the last exchange closes, which means there will always be a bar even if there are no events during the bar period.

Carrying Forward of Current NBBO Bid/Ask

If there are no changes to the Bid/Ask in the NBBO during a bar period, the current NBBO Bid/Ask from the previous bar period will be carried forward.

Quote Price Filter

When trading and quoting activities are inactive, for example, during extended trading hours or with an illiquid stock, bid prices can be extremely low, and ask prices can be extremely high. An exchange can also send a bad price, for example, a stock has a bid of $12.05 then an exchange sends a bid of $212.05.

To make this Minute Bar dataset usable for illiquid stocks and ETFs/ETNs, algoseek filters out extreme quotes by the following two criteria:

Bid price < (0.05 x average price of last 10 days)

Ask price > (10 x average price of last 10 days)

Separating Exchange and Finra Volume

The bars have trade volume separated into fields:

ExchangeVolume: Trades done on the listed exchanges

FinraVolume: Trades done in Dark Pools, internally by Broker-Dealers, or on an Over-the-Counter (OTC) market reporting to FINRA

The volume data is separated to make it easy to understand the trading in a bar period for either the public-listed exchanges or private non-public trading.

DATA ORGANIZATION AND FILE FORMAT

algoseek provides Equity market data in plain text CSV files. The first row of CSV file is a fixed header and then rows of data corresponding to individual bars. By default, data is organized into one file per symbol per trading day.  For example, all trade and quote bars for ticker AAPL on Mar 3, 2020, are stored in one CSV file.  

Due to their large data sizes, CSV files are gzip-compressed (with a csv.gz extension) with a compression ratio of about 8:1.

Table 1 demonstrates the full list of data fields in the Equity Buy/Sell Pressure and Retail Indicators file with sample contents for the AAPL symbol.

Table 1: Equity Buy/Sell Pressure and Retail Indicators Sample Data

Table 2 (below) provides the name, base event, default value, brief description, and data type for each data field (column) in the Equity Buy/Sell Pressure and Retail Indicators Minute Bar CSV file.

Table column “Missing” indicates a default behavior in case the data field value is not present or cannot be calculated. The column value “Never” means that the data field value is always present.

The table column “Base Event” indicates what type of events are included for data field calculation. Quote: bid/ask event, Trade-X: trades on the exchange, Trade-F: trades on FINRA/TRF, Trade: trades on both exchange and FINRA/TRF.

Table 2: Equity Buy/Sell Pressure and Retail Indicators CSV File Fields Schema

Time Range

The Buy/Sell Pressure and Retail Indicators Minute Bar dataset covers the entire trading day from the start of pre-market trading to the end of after-hours trading (ET time):

Pre-Market Hours: 04:00:00 to 09:30:00 (excluding)

Market Hours: 09:30:00 to 16:00:00 (excluding)

Post-Market Hours: 16:00:00 to 20:00:00

Note: Occasionally, minute bars are extended several minutes past 20:00.

Market Holidays and Early Closes

The stock market is closed for trading on most US holidays. For reference, algoseek publishes a list of historical holidays which is available at s3://us-equity-market-holidays/holidays.csv (direct download link: https://us-equity-market-holidays.s3.amazonaws.com/holidays.csv).

Markets sometimes close early at 13:00:00 on the day before holidays such as Independence Day and Thanksgiving. You can download algoseek’s early close date and time list from AWS S3 storage at s3://us-equity-market-holidays/earlycloses.csv (or use a direct link: us-equity-market-holidays.s3.amazonaws.com/earlycloses.csv).

Bar Notes

Time Bar Start Format: One-second bar 13:03:01 is from time greater than 13:03:01 to less than 13:03:02. One-minute bar 11:04 is from time greater than 11:04 to less than 11:05.

Empty Fields: an empty field has no value and is “Blank.” For example, “ExchangeTradeCount” and there are no trades during the bar period. The field “TotalTrades” measuring the total number of trades in a bar will be “0” if there are no Trades. Look at the “Missing” column above for each field.

Spread Validation Rules: For fields that include spreads (for example, VolumeWeightSpread), the spread needs to be excluded when it is clearly incorrect.

Pre-Market: Exclude when a bid or ask is further away than 30% of the midpoint.  For example, if stock is 100 then

bid >= (0.7 * 100) = 70 and Ask <= (1.3 * 100) = 130

Regular Market: Exclude when a bid or ask is further away than 10% of the midpoint.  For example, if stock is 100 then

bid >= (0.9 * 100) = 90 and Ask <= (1.1 * 100) = 110

Use dynamic calculation to move from Pre-Market to Regular Market. When 3 or more Bid/Ask spreads are within 10% of the midpoint after the start of the Regular market (9:30:00), continue to use 10% of the midpoint.  After 20 NBBO updates (consider one NBBO update to be two rows with an update for Bid and Ask), move to 10%.

Post Market: Exclude when a bid or ask is further away than 30% of the midpoint. Start immediately after market Close (4 pm or 1 pm for half-days)

Always exclude spread for

• Cross/Locked: If NBBO is crossed/locked, then exclude.
• Missing Bid/Ask:  If an NBBO update is missing a Bid/Ask, then ignore the update.

The field “SpreadValidTime” has the total milliseconds for each bar period showing the total number of milliseconds that meet valid criteria based on the requirements listed here.

RelativeSpreadAverage: The relative spread size is the Bid/Ask spread relative to the midpoint price at time t for a trade.  It shows how wide the spread is compared to the midpoint price.  For each minute, the average of the relative spread sizes for each trade is calculated.

For each trade in the bar:

t = time of a trade

b = National Best Bid (NBBO) at t (use last NBBO before time t)

a = National Best Ask (NBBO) at t (use last NBBO before time t)

m = midpoint price at t, m = (b + a) / 2

rs = relative spread at t is max(a - b, 0) / m (max function is used as NBBO spread may be 0 or inverted at times)

RelativeSpreadAverage = sum(rs) / count(rs)

TradeCumulDistributionToBid: A distribution of trades relative to the bid and offer. Calculate the distance of each trade to the Bid with 0 executed at the Bid and 1 executed at the Ask. Then calculate the cumulative distribution of the distance from the Bid using the below probabilities.

0, 0.05, 0.1, 0.20, 0.40, 0.60, 0.80, 0.90, 0.95, 1

This shows if there was pressure on either side of the bid offer and if ‘retail trade’ moves away from the paper – there will be a potential way to recalibrate it without recalculating everything in the history again. As an example: say 1000 shares are traded in the given bin. 100 at a bid, 400 at mid, and 500 at the offer. The distribution above will look as follows

 0       100

 0.05    100

                                           0.1     100

 0.2     100

 0.4     100

 0.6     500

 0.8     500

 0.9     500

 0.95    500

 1       1000

RetailTRFBuySize, RetailTRFSellSize: Identify Retail Buy and Sell trades executed internally by a Broker-Dealer or wholesale to dark-pool based on their sub-penny pricing as a TRF reported trade.  These indicators are based on the research paper “Tracking Retail Investor Activity” by  E. Boehmer, Ch. M. Jones, X. Zhang, and X. Zhang published in the Journal of Finance (Tracking Retail Investor Activity).

The field calculations are based on the following description from the paper:

Based on these institutional arrangements, identifying transactions initiated by retail customers is fairly straightforward. Transactions with a retail seller tend to be reported on a TRF at prices that are just above a round penny due to the small price improvement, while transactions with a retail buyer tend to be reported on a TRF at prices just below a round penny. To be precise, for all trades reported to a FINRA TRF (exchange code “D” in TAQ), let Pit be the transaction price in stock i at time t, and let Zit ≡ 100 * mod(Pit, 0.01) be the fraction of a penny associated with that transaction price. Zit can take any value in the unit interval [0,1). If Zit is in the interval (0,0.4), we identify it as a retail sell transaction. If Zit is in the interval (0.6,1), then the transaction is coded as a retail buy transaction. To be conservative, transactions at a round penny (Zit = 0) or near the half-penny (0.4 ≤ Zit ≤ 0.6) are not assigned to the retail category.

• Transactions at or near the half-penny are not assigned to the retail category because, in this case, they are likely to be from institutions.
• Also, transactions with a $100k dollar-volume and higher are excluded as they can’t be retail trades.
• Skip transactions for spreads other than 1 cent.

RetailOddLotBuySize, RetailOddLotSellSize: Identify Retail Buy and Sell trades from odd lots transactions executed on public exchanges. These indicators are based on a comparison of trade price with the NBBO midpoint. If the trade price is higher than the midpoint the transaction is identified as a buy transaction. If it is lower than the midpoint the transaction is identified as a sell transaction.

Additional Indicators

1. TRF Retail Pressure
   Def: This metric measures the proportion of TRF retail trades’ size to total market volume.

RP = (RetailTRFBuySize + RetailTRFSellSize) / TotalVolume

2. Odd Lot Pressure
   Def: This metric measures the proportion of odd lot (on public exchanges) trades’ size to total market volume.
   
   OLP = (RetailOddLotBuySize + RetailOddLotSellSize) / TotalVolume
3. TRF Retail + Odd Lot Pressure
   Def: This metric measures the proportion of TRF retail trades’ size plus odd lot (on public exchanges) trades’ size to total market volume.
   
   ROLP = (RetailTRFBuySize + RetailTRFSellSize + RetailOddLotBuySize + RetailOddLotSellSize) / TotalVolume
4. Odd Lot TRF Retail Ratio

Def: This metric measures the odd lot (on public exchanges) volume compared to the total TRF retail plus odd lot (on public exchanges) volume.

OLRR = (RetailOddLotBuySize + RetailOddLotSellSize) / (RetailTRFBuySize + RetailTRFSellSize + RetailOddLotBuySize + RetailOddLotSellSize)

5. TRF Retail Buy-Sell Ratio
   Def: This metric measures the TRF retail buy size to the TRF retail sell size.
   
   RBSR = RetailTRFBuySize / RetailTRFSellSize
6. Odd Lot Buy-Sell Ratio
   Def: This metric measures the odd lot (on public exchanges) buy size to the odd lot (on public exchanges) sell size.
   
   OLBSR = RetailOddLotBuySize / RetailOddLotSellSize
7. TRF Retail + Odd Lot Buy-Sell Ratio
   Def: This metric measures the TRF retail plus odd lot (on public exchanges) buy size to the TRF retail plus odd lot (on public exchanges) sell size.
   
   ROLBSR = (RetailTRFBuySize + RetailOddLotBuySize) / (RetailTRFSellSize + RetailOddLotSellSize)
8. Relative Net TRF Retail Flow
   Def: This metric measures the relative net difference between TRF retail buy and sell sizes to total TRF retail volume. A positive RNRF suggests buying pressure from retail investors, while a negative RNRF indicates selling pressure.
   
   RNRF = (RetailTRFBuySize - RetailTRFSellSize) / (RetailTRFBuySize + RetailTRFSellSize)
9. Relative Net Odd Lot Flow
   Def: This metric measures the relative net difference between odd lot (on public exchanges) buy and sell sizes to total odd lot (on public exchanges) volume. A positive RNOLF suggests buying pressure from retail investors, while a negative RNOLF indicates selling pressure.
   
   RNOLF = (RetailOddLotBuySize - RetailOddLotSellSize) / (RetailOddLotBuySize + RetailOddLotSellSize)
10. Relative Net TRF Retail + Odd Lot Flow

Def: This metric measures the relative net difference between TRF retail plus odd lot (on public exchanges) buy and sell sizes to total TRF retail plus odd lot (on public exchanges) volume. A positive RNRODF suggests buying pressure from retail investors, while a negative RNRODF indicates selling pressure.

RNRODF = ((RetailTRFBuySize + RetailOddLotBuySize) - (RetailTRFSellSize + RetailOddLotSellSize)) / (RetailTRFBuySize + RetailTRFSellSize + RetailOddLotBuySize + RetailOddLotSellSize)

11. TRF Retail Imbalance
    Def: This metric highlights the imbalance between TRF retail buy and sell sizes. An RI value closer to 1 indicates a strong buy-side dominance, while a value closer to 0 indicates sell-side dominance. A value around 0.5 suggests balanced TRF retail activity.
    

RI = RetailTRFBuySize / (RetailTRFBuySize + RetailTRFSellSize)

12. Odd Lot Imbalance
    Def: This metric highlights the imbalance between odd lot buy (on public exchanges) and sell (on public exchanges) sizes. An OLI value closer to 1 indicates a strong buy-side dominance, while a value closer to 0 indicates sell-side dominance. A value around 0.5 suggests balanced odd lot activity.

OLI = RetailOddLotBuySize / (RetailOddLotBuySize + RetailOddLotSellSize)

13. TRF Retail + Odd Lot Imbalance
    Def: This metric highlights the imbalance between TRF retail plus odd lot buy (on public exchanges) and TRF retail plus odd lot sell (on public exchanges) sizes. A ROLI value closer to 1 indicates a strong buy-side dominance, while a value closer to 0 indicates sell-side dominance. A value around 0.5 suggests balanced activity.
    
    ROLI = (RetailTRFBuySize + RetailOddLotBuySize) / (RetailTRFBuySize + RetailTRFSellSize + RetailOddLotBuySize + RetailOddLotSellSize)
14. TRF Retail Sentiment

Def: A sentiment indicator that measures TRF retail pressure 1-hour cycles based on buy/sell imbalances.

RS = MovingAverage(Relative Net TRF Retail Flow)

15. Odd Lot Sentiment
    Def: A sentiment indicator that measures odd lot pressure (on public exchanges) 1-hour cycles based on buy/sell imbalances.
    
    OLS = MovingAverage(Relative Net Odd Lot Flow)
16. TRF Retail + Odd Lot Sentiment
    Def: A sentiment indicator that measures TRF retail plus odd lot pressure (on public exchanges) 1-hour cycles based on buy/sell imbalances.
    
    ROLS = MovingAverage(Relative Net TRF Retail + Odd Lot Flow)

APPENDIX A. FREQUENTLY ASKED QUESTIONS

Why are time-based columns not properly recognized when I try importing data to Excel?

Older versions of Excel will automatically convert the TimeBarStart field into an Excel format timestamp, but this fails when TimeBarStart is HHMMSSmmm (millisecond) or HHMMSSmmmiiinnn (nanosecond). For timestamp with the nanosecond (millisecond) format, import the data using the Excel “From Text” option and set the data type for column “TimeBarStart” to “Text”, so Excel does not automatically try to convert it.

APPENDIX B. BAR CALCULATIONS FROM TRADE AND QUOTE EVENTS

This section describes logic for minute bar calculations based on events from the Trade and Quote dataset. Please also refer to the Equity Trade and Quote Guide for more details on data fields and condition flags used.

There is a separate logic for the Standard Bars dataset and Bars with FINRA/TRF and Odd Lots Excluded.

Standard Trade and Quote Minute Bar

Excluded data

• Exclude any trade with Price = 0 (not applied to Trade Cancel)
• Exclude any trade with Quantity = 0
• Exclude any quote with Price = 0
• Exclude any quote with Quantity = 0

You should also exclude any event with one or more flags listed in Table 2.

Table 2: Flags for Trade and Quote Events to be Excluded During Bar Calculations

Included data

• Include only NBBO quotes
• Include both NBBO and exchange trades
• Include Trade Cancel events only for the CancelSize field (no price or flag filter applied)

You should only include events with one or more flags listed in Table 3. If the event has any of the exclude flags enabled, it is not included. If the event does not contain any flags from the include list, it is not included in bar calculations.

Table 3:  Flags for Trade and Quote Events to be Included During Bar Calculations

Price validation

Additionally, you should filter out test quote events using the following approach:

1. For symbols with price history (last 10 trading days):

MinPrice = 0.05 * AveragePrice

MaxPrice = 10 * AveragePrice

2. For new symbols (no price history):

MinPrice = 0.03

MaxPrice = 19998

If a Quote price is lower than MinPrice or higher than MaxPrice - the event is excluded.

Note: We do not recommend applying price filtering for Trade events.