Edited by Dheeraj Singh.
Bayesian and Time Series…Why Not Both?
Imagine peering into the future, not with a dusty crystal ball, but with the symphony of data at our fingertips. In the realm of data science, we don’t need to rely on mind readers or magical artefacts to foresee the future. Can we truly predict the next pandemic or the next market downfall? This is where the power of time series analysis meets the elegance of Bayesian inference, and working in harmony, they create a future with endless possibilities. (Hansen, 2023)
Understanding Time Series
Time series data consists of observations recorded sequentially over time. Unlike other data types, time series data exhibits temporal dependencies, meaning current values are influenced by past values. At its core, time series data is characterised by autocorrelation (Smith & Murry, 2023) where observations are not isolated, but rather influenced over time. For example, imagine Malaysia's boiling weather every day: a hot day is more likely to be followed by another hot day compared to a random observation.
These data sets could exhibit trends both upward or downward as well as seasonality or cyclical patterns. Techniques such as differencing (removing trends) and seasonal decomposition are employed to prepare the data for analysis. (Brownlee, 2020)
The Power of Bayesian Inference
Now, you may ask - why should we bother with Bayesian Inference at all when we have our traditional forecasting models like the ARIMA (Autoregressive Integrated Moving Average)? Unfortunately, the real world throws curveballs, nonlinearities and intricate dynamics that can confound these models. Enter Bayesian inference, a statistical game-changer that tackles these challenges.
Bayes’ inference is based on Bayes’ theorem, which provides a mathematical framework for updating the probability of a hypothesis as new data becomes available. Prior distributions represent our beliefs about the parameters before observing the data. (Adam Hayes, 2024). As new data is collected, these priors are updated to form posterior distributions, as seen in the image below. By having a process where beliefs are constantly updated, this makes Bayesian inference powerful for time series forecasting, where incorporating prior and current information can significantly enhance the model’s accuracy.
A simple example would be to imagine a Bayesian VAR (Vector Autoregression Model), a statistical model not only used to forecast multiple time series variables that might influence each other, but also used to forecast the relationship between multiple economic indicators. As new economic data becomes available, Bayes’ theorem allows us to update these priors, resulting in a posterior distribution that directly reflects our current understanding of the relationships between these economic variables.
Advanced Bayesian Techniques Used
There are many other techniques that are used along with the Bayesian theorem to ensure an accurate model - after all, it is incredibly difficult to predict the exact probability of all possible parameter values.
If we are dealing with simpler time series models, we could use the concept of conjugate priors. (Aerin, 2020). With the use of normal distribution, the prior normal distribution would have a specific mean μ₀ and standard deviation σ₀ based on an initial guess. For example, if we’re trying to estimate the average monthly rainfall μ, we will have a likelihood function that describes the probability of observing the actual rainfall data points (y₁, y₂, ..., yₙ). Bayes’ theorem allows us to update our initial belief after considering the data which contains the likelihood of an event. However, sometimes we may need more complex methods for more complex time series models.
One of the most significant ones is the Markov Chain Monte Carlo (MCMC) method, which is a computational technique used to estimate complex probability distributions, like the posterior distribution. (Liu, n.d.)
Imagine a blindfolded person standing in the middle of the room. This person will represent a possible parameter in the model. Using the existing data and the model’s structure, this method allows the person to take tiny, random steps around the room, which, with regards to the model, is exploring different parameter values. With each step, the person gathers more information about the room, thus they are eventually able to paint an accurate image of the room. This represents the distribution of the posterior distribution. Naturally, the more steps we take, the more accurate the model becomes.
Integration of Bayesian Methods with Time Series Models
Bayesian Inference can be seamlessly integrated with time series models such as the ARIMA models in countless different industries, ranging from healthcare, finance, construction to many others. A popular example of when this was done was during COVID-19, where Bayesian ARIMA models were used to forecast infection rates (Tan, 2022), incorporating prior information about disease transmission dynamics and continually updating predictions as new data was received. In practice, Bayesian ARIMA models can be implemented using statistical software such as R. Below is an example code demonstrating the implementation of a Bayesian ARIMA model in R:
library(rstanarm)
# Example data: Monthly airline passengers
data <- AirPassengers
# Fit Bayesian ARIMA model
model <- stan_arima(data, order = c(1, 1, 1))
# Forecast future values
forecast <- posterior_predict(model, horizon = 12)
print(forecast)
Furthermore, in the field of finance, a case study using Bayesian methods to forecast the stock prices during the 2008 financial crisis highlighted the benefits of this approach, as it allowed active updating of models in response to rapidly changing market conditions.
Conclusion and What the Future Holds
The future holds countless possibilities for the seamless integration of Bayesian methods and advanced time series models with the most exciting ones being the application of Bayesian methods to new domains such as climate modelling and epidemiology. Researchers are also finding new ways to improve computational efficiency and scalability, making Bayesian methods much more accessible for large-scale time series analysis. As research and development continue, Bayesian time series analysis will play an increasingly important role in various domains, driving innovation and improving decision-making processes.
References:
Adam Hayes. (2024, March 20). Bayes' Theorem: What It Is, Formula, and Examples. Investopedia. Retrieved July 15, 2024, from https://www.investopedia.com/terms/b/bayes-theorem.asp
Aerin. (2020). Conjugate Prior Explained. With examples & proofs | by Ms Aerin. Towards Data Science. Retrieved July 16, 2024, from https://towardsdatascience.com/conjugate-prior-explained-75957dc80bfb
Brownlee, J. (2020, June 23). How to Remove Trends and Seasonality with a Difference Transform in Python - MachineLearningMastery.com. Machine Learning Mastery. Retrieved July 15, 2024, from https://machinelearningmastery.com/remove-trends-seasonality-difference-transform-python/
Hansen, M. W. (2023, May 14). Bayesian Structural Time Series and ARIMA; why not both? Medium. Retrieved July 15, 2024, from https://medium.com/@martin_wh/bayesian-structural-time-series-and-arima-why-not-both-de7fcd163d37
Illustration of Markov Chain Monte Carlo method | Download Scientific Diagram. (n.d.). ResearchGate. Retrieved July 16, 2024, from https://www.researchgate.net/figure/Illustration-of-Markov-Chain-Monte-Carlo-method_fig1_334001505
Liu, J. (n.d.). Markov Chain Monte Carlo. Columbia University Mailman School of Public Health. Retrieved July 16, 2024, from https://www.publichealth.columbia.edu/research/population-health-methods/markov-chain-monte-carlo
Relationship between Bayesian prior, posterior, and data. Prior... | Download Scientific Diagram. (n.d.). ResearchGate. Retrieved July 16, 2024, from https://www.researchgate.net/figure/Relationship-between-Bayesian-prior-posterior-and-data-Prior-knowledge-and-insight-is_fig1_254087538
S, R. (2023, April 25). Time Series Analysis: Understanding Seasonality and Cyclicality | by Rahul S | Medium. Rahul S. Retrieved July 16, 2024, from https://ogre51.medium.com/in-time-series-forecasting-what-do-you-think-is-the-difference-between-seasonality-and-cyclicity-f4e8d9523d24
Smith, T., & Murry, C. (2023). Autocorrelation: What It Is, How It Works, Tests. Investopedia. Retrieved July 15, 2024, from https://www.investopedia.com/terms/a/autocorrelation.asp
Tan, F. (2022, December 30). Comparing COVID-19 Case Prediction Between ARIMA Model and Compartment Model — China, December 2019–April 2020. China CDC Weekly. Retrieved July 16, 2024, from https://weekly.chinacdc.cn/en/article/doi/10.46234/ccdcw2022.239
Comments