Using Multivariate Functional False Discovery Rate for Testing System Robustness

Data snooping bias is a prevalent challenge in finance, stemming from the extensive exploration of historical financial data to uncover trading strategies and patterns. It occurs when researchers or traders test numerous hypotheses and strategies on historical data until they discover one that appears profitable purely by chance. This process, while inadvertently identifying a seemingly effective strategy, ignores the inherent randomness in financial markets. Subsequently, when the strategy is applied to new, unseen data, its historical performance may not be replicable due to chance findings in the initial data snooping process. Data snooping bias can lead to the development of overly optimistic expectations and flawed trading strategies that perform poorly in real-market conditions, posing a significant risk to investors and traders who fail to account for this bias in their decision-making processes.

Reference [1] proposed the use of a multivariate functional false discovery rate (mfFDR) method as a solution to address the issue of data snooping bias within the framework of designing trading systems. This approach offers a systematic method for selecting robust trading strategies, effectively countering the pitfalls associated with data snooping bias. The author pointed out,

Empirically, we apply the mfFDR on a large universe of technical trading rules to detect truly profitable ones with control of data snooping. We first study the trading rules on each currency individually. With the use of multiple informative covariates, the results show that the mfFDR based portfolio is much more powerful than the existing methods that are not using covariates. After combining, the portfolio of selected rules can generate positive profit, which is higher than the data snooping control methods with and without using a sole covariate. We then study 30 currencies together where more than 600 thousand trading rules are generated. We implement the method on this set of rules to construct a portfolio that generates a Sharpe ratio of roughly one for roughly 50 years. Given that the profitable portfolio of trading rules is constructed based on only past exchange spot rate data, the results show the value of technical rules in the foreign exchange market. Further analyses additionally provide insight information on which categories of trading rule traders should take into consideration when constructing a portfolio of trading rules.

It is commendable that research addresses the matter of trading system robustness. However, we observe that the proposed method does not necessitate the utilization of out-of-sample data, which is somewhat unexpected. In essence, the testing procedures were exclusively conducted using in-sample data. The question arises as to why these trading systems exhibited favorable performance in out-of-sample scenarios. Could this be attributed to the persistence of profit and loss in the FX market, as indicated by the author? Moreover, can these systems maintain their profitability when applied to equity trading?

…the performance of technical trading rules in the foreign currency market is persistent. Consequently, by investing on truly profitable strategies in a in-sample period, an investor should be able to generate an out-of-sample (OOS) profit.

Let us know what you think in the comments below or in the discussion forum.

References

[1] Tren Ma, Controlling for Luck in Picking Outperforming Trading Strategies, 2023

Article Source Here: Using Multivariate Functional False Discovery Rate for Testing System Robustness

source https://harbourfronts.com/multivariate-functional-false-discovery-rate-for-testing-system-robustness/