Learning Objectives

• Understand how to visualize and explore financial time series data
• Learn key R packages and functions for financial analysis
• Develop intuition for patterns in financial data

Why This Matters

• Good data exploration helps identify potential modeling approaches
• Visual analysis reveals patterns that statistics might miss
• Essential skill for any financial analyst

Outline

• package tsfe and class datasets
• ts objects and ts function
• Visual trends and patterns
• Distributions properties of asset returns
• Time series decomposition

Rethinking visualisation

• Charts are not meant to be seen, they are intended to be read
• They are not just images but visual arguments.
• Data doesn’t speak for itself.
• Data visualisations need to be shown and explained

ts objects and ts function

A time series is stored in a ts object in R:

• a list of numbers
• information about times those numbers were recorded.

Example

x <- c(123,39,78,52,110)
yr <- 2012:2016
knitr::kable(data.frame(Year=yr,Observation=x), booktabs=TRUE)

Some class data

MyDeleteItems<-ls() 
rm(list=MyDeleteItems) 
# Good practice to clear all objects before loading data
library(tsfe) # includes class datasets
print(data(package='tsfe'))

Programmatically accessing data from the

# Download FTSE 100 index data starting from 2016
ftse <- tq_get("^FTSE", from="2016-01-01")

# Convert daily data to monthly frequency
# tq_transmute is used to transform the data while selecting specific columns
ftse_m <- tq_transmute(ftse,
                     select = adjusted,    # Select adjusted closing prices
                     mutate_fun = to.monthly)  # Convert to monthly frequency

# Create a time series object from the monthly data
# Start date is set to January 2016 with monthly frequency (freq=12)
ftse_m_ts <- ts(ftse_m$adjusted, 
                start=c(2016,1),  # Starting year and month
                freq=12)          # 12 periods per year (monthly data)

autoplot(tsfe::ftse_m_ts) + theme_tq()

Is a table a good visualisation?

• Quarterly earnings per share data for carnival PLC.

• Using DT to create an interactive table

#|echo: true

tsfe::ni_hsales %>% datatable()

Your turn

• Use tq_get to download the CBOE VIX Index from 2016-01-01 using the symbol ^VIX
• create a time series object using the VIX index price.
• Plot this daily VIX Index Price using autoplot.

Hint: the ts object of daily financial time series does not have a regular frequency to input into ts() function, so leave this argument blank.

Why normal?

• Named after Carolo Friderico Gavss, the normal (or Gaussian) distribution is the most common used distributional assumption in statistical analysis.

This is because:

• Easy to calculate with
• Common in nature
• Very conservative assumptions

the importance of simulation

• .Second, we can use simulation to approximate the sampling distribution of data and propagate this to the sampling distribution of statistical estimates and procedures.

• Third, regression models are not deterministic; they produce probabilistic predictions.

the importance of simulation

• Simulation is the most convenient and general way to represent uncertainties in forecasts.

Throughout this course and in our practice, we use simulation for all these reasons;

• In this lecture we introduce the basic ideas and the tools required to perform simulations in R. ]

Monte Carlo Simulation

• The coding for simulation becomes cleaner if we express the steps for a single simulation in an R function.

# Function to simulate a single price path
path_sim <- function(){
    days <- 200                           # Simulation horizon
    
    # Generate daily returns with slight upward drift
    # mean=1.001 implies 0.1% average daily return
    # sd=0.005 represents daily volatility of 0.5%
    changes <- rnorm(200, mean=1.001, sd=0.005)
    
    # Calculate cumulative price path starting from £40
    # cumprod() calculates the cumulative product of the changes
    sample.path <- cumprod(c(40, changes))
    
    # Return the final price after 200 days
    closing.price <- sample.path[days+1]  # +1 because we add the opening price
    return(closing.price)
}

# Run the simulation 10,000 times to generate possible future prices
number_of_possible_futures <- 10000
mc.closing <- replicate(number_of_possible_futures, path_sim())

Key Improvements in Enhanced Simulation

• More realistic assumptions:
  • Uses actual trading days (252) instead of calendar days
  • Creates fat tails using mixture of normal distributions
  • Annual parameters based on typical market behavior
  • Models more extreme market events
• Uses log-returns instead of simple returns
• Provides more comprehensive risk assessment

Summarising simulations

• Simulations are a versatile way to summarise a probability model, predictions from a fitted regression or uncertainty about parameters of a fitted model (the probabilistic equivalent of estimates and standard errors)
• One useful way to summarise the location of the distribution is to use the median function in the variation in the distribution is the median absolute deviation standard deviation (mad sd).

Median statistics

We typically prefer median-based summaries because they are more computationally stable, and we rescale the median-based summary of variation as described above so as to be comparable to the standard deviation, which we already know how to interpret in usual statistical practice.

cat("median = ",median(mc.closing),
    "mad sd = ",mad(mc.closing),
    "mean = ",mean(mc.closing),
    "sd = ",sd(mc.closing))

median =  48.75303 mad sd =  3.441515 mean =  48.85153 sd =  3.466317

Why normal?

Practical reasons

• Processes that produce normal distributions include:
• Addition of many independent random variables
• Lots of process are approximately normal
• Product of small deviations
• logarithms of products
• Logarithms are just magnitudes

Understanding Normal Distributions: Two Perspectives

Why These Perspectives Matter in Finance

• Ontological (Reality):
  • Helps understand how market prices actually form
  • Explains why some patterns naturally emerge
  • Shows limits of our ability to predict
• Epistemological (Knowledge):
  • Guides our modeling choices
  • Helps avoid overconfident predictions
  • Explains why we use normal distributions even when we know they’re not perfect

Why normal asset returns?

• Financial time series processes considered in this course:
• Return series
• Financial statement information
• Volatility processes
• Extreme events
• Multivariate series

Simple model of asset returns

• To engineer a statistical model of asset returns we need to make some assumptions about the data story or data generating process.

\[ \{R_{it}|i-1,\dots N;t=1,\dots,T\} \stackrel{i.i.d}\sim N (m_1,m_2) \]

• A traditional assumption in financial analysis is the simple returns are independently and identically distributed (iid) as normal with a fixed mean \(m_1\) and variance \(m_2\).

Another model of asset returns

\[ \{r_{it}|i=1,\dots N ;t=1,\dots,T\} \stackrel{i.i.d}\sim N (\mu,\sigma^2) \]

• Another common assumption is that log returns are iid as normal with mean \(\mu\) and variance \(\sigma^2\).
• As the sum of a finite number of iid random variables is normal, \(r_t[k]\) is also normally distributed.
• There is also no lower bound for \(r_t\)
• However, lognormal assumption is not consistent with positive excess kurtosis.

Are stock returns distributed normally?

Visual Arguments

• We can use ggplot to visualise the empirical distribution, superimposing what the returns would look like if they were normally distributed.

  • Only two parameters (a mean and a variance) are required to create a hypothetical normal distribution of a returns series

ggplot code

# Download Glencore stock data from Yahoo Finance
glen <- tq_get("GLEN.L")

# Calculate monthly log returns
glen_m <- glen %>%
    tq_transmute(
        select = adjusted,           # Select adjusted closing prices
        mutate_fun = monthlyReturn,  # Convert to monthly returns
        type = "log",               # Calculate log returns
        col_rename = "log_return"    # Name the new column
    )

# Create a time series object from the returns
# Starting from May 2011 with monthly frequency
glen_m_ts <- glen_m$log_return %>%
    ts(frequency=12, start=c(2011,5))

# Create density plot comparing returns to normal distribution
glen_m %>% 
    ggplot(aes(x=log_return)) +
    geom_density() +                # Plot the actual return distribution
    stat_function(                  # Add normal distribution line
        fun=dnorm,
        args=list(
            mean(glen_m$log_return),    # Use actual mean
            sd=sd(glen_m$log_return)    # Use actual standard deviation
        ),
        col="red"
    )

Inference from the plot

• What patterns are revealed?
• The normal distribution is superimposed over the histogram of the daily equity returns.
• Compared to the normal the distribution of the returns has longer tails and a higher central peak.
• In statistical terms we say the distribution is leptokurtic, or fat-tailed.

Inference from plot

• This plot compares the quantiles of a normal distribution (thinner straight line) to the quantiles of the data (thicker scatter plot).
• If the plots exactly overlap then the data is probably normally distributed.
• While the returns and the normal distribution are similar between +12.5% and -12.5%, outside these limits the returns behave non-normally.

Should we care if asset returns are normal distributed variables?

• In regressions the assumption of normality of model errors is one of the least important
• For the purpose of estimating the regression line it is barely important at all
• Diagnostics of normality of errors is not recommended unless the model is being used to predict individual data points.

Heavy Tail Statistical Distributions

Student’s t

very similar to the normal but wider and lower.

Stable

generalisation of the normal, stable under addition thus can be used with log returns, can capture excess kurtosis well but has infinite variance which conflicts with finance theory.

Scale mixture

a combination of a number of normals.

Rethinking time plots

• One limitation with time plots is that the simple passage of time is not a good explanatory variable.
• There are occasional exceptions where there is a clear mechanism driving the financial time series.

Descriptive chronology is not causal explanation - Edward Tufte ( 2015) The visual display of quantitative information P37

Time plot

glen_m %>% ggplot(aes(x=date,y=log_return)) + 
  geom_line()

Seasonal plots

glen_m_ts %>% ggseasonplot(year.labels=TRUE,
                           year.labels.left=TRUE) + 
  ylab("") +
  ggtitle("Seasonal plot: Glencore returns")

Inference from the plot

• Data plotted against the individual “seasons” in which the data were observed. (In this case a “season” is a month.)
  • Something like a time plot except that the data from each season are overlapped.
  • Enables the underlying seasonal pattern to be seen more clearly, and also allows any substantial departures from the seasonal pattern to be easily identified.
  • In R: ggseasonplot()

Polar plot

glen_m_ts %>% ggseasonplot(polar=TRUE) + ylab("")

Seasonal or cyclic?

Time series patterns

Trend

pattern exists when there is a long-term increase or decrease in the data.

Seasonal

pattern exists when a series is influenced by seasonal factors (e.g., the quarter of the year, the month, or day of the week).

Cyclic

pattern exists when data exhibit rises and falls that are not of fixed period (duration usually of at least 2 years).

Time series components

Differences between seasonal and cyclic patterns:

• seasonal pattern constant length; cyclic pattern variable length
• average length of cycle longer than length of seasonal pattern
• magnitude of cycle more variable than magnitude of seasonal pattern

Time series patterns

ts(tsfe::ni_hsales$`Total Verified Sales`,start = c(2005,1),frequency = 4)->ni_hsales_ts
autoplot(ni_hsales_ts) +
  ggtitle("Northern Ireland Quarter House Sales") +
  xlab("Year") + ylab("Total Verified Sales")

Time series patterns

autoplot(carnival_eps_ts) +
  ggtitle("Quarterly EPS for Carnival Plc") +
  xlab("Year") + ylab("")

Time series patterns

tsfe::usuk_rate %>%
  ggplot(aes(x=date,y=price)) +
  geom_line(colour="darkgreen") +
  labs(title=" Time Plot of GBP:USD",
       x="",
       y="Value of £1 in Dollars")

Autocorrelation

Results for first 9 lags for Carnival earnings data:

ACF plots Trend and seasonality

• When data have a trend, the autocorrelations for small lags tend to be large and positive.
• When data are seasonal, the autocorrelations will be larger at the seasonal lags (i.e., at multiples of the seasonal frequency)
• When data are trended and seasonal, you see a combination of these effects.

Any trends?

autoplot(carnival_eps_ts)

Any autcorrelation?

ggAcf(carnival_eps_ts)

Your turn

• use tq_get() to download GLEN.L stock price from yahoo finance
• using this series create a time series object hint use frequency=1
• Explore this time series for trends and autocorrelation

Example: White noise

set.seed(1222)
wn <- ts(rnorm(36))
autoplot(wn)

Example: White noise

• Sample autocorrelations for white noise series

• We expect each autocorrelation to be close to zero.

Sampling distribution of autocorrelations

Sampling distribution of \(r_k\) for white noise data is asymptotically N(0,\(1/T\)).

• 95% of all \(r_k\) for white noise must lie within \(\pm 1.96/\sqrt{T}\).
• If this is not the case, the series is probably not WN.
• Common to plot lines at \(\pm 1.96/\sqrt{T}\) when plotting ACF.

Your turn

• You can compute the daily changes in the Google stock price using

dgoog <- diff(goog)

• Does dgoog look like white noise?

Time Series Modelling

• Financial time series data often exhibit patterns, trends, and fluctuations
• Appropriate modelling and processing techniques are required to extract meaningful insights
• Two commonly used approaches:
  • ARIMA (Autoregressive Integrated Moving Average) modelling
  • Smoothing techniques

ARIMA Modelling

• Class of statistical models widely used for time series forecasting and analysis
• Combines autoregressive (AR) and moving average (MA) components
• Handles non-stationarity through differencing

Suitability of ARIMA Models

• Capturing underlying patterns and dynamics of time series data
• Handling trends, seasonality, and autocorrelation structures
• Widely used in finance for forecasting stock prices, exchange rates, and economic indicators

Smoothing Techniques

• Reduce noise or irregularities in time series data
• Reveal the underlying trend or signal
• Apply filters or weighted averages to smooth out fluctuations
• Do not explicitly model the autocorrelation structure

Suitability of Smoothing Techniques

• Extracting the underlying trend or signal from noisy data
• Preprocessing step before further analysis or visualization
• Focus on denoising the data and revealing the underlying trend or signal

Choosing the Right Approach

• Depends on the specific objectives and characteristics of the financial time series data
• ARIMA models:
  • Forecasting future values
  • Accounting for autocorrelation and capturing underlying patterns
• Smoothing techniques:
  • Denoising the data and revealing the underlying trend or signal

Combining Approaches

• ARIMA modelling and smoothing techniques can be combined
• Used in conjunction with other techniques (decomposition methods, machine learning algorithms)
• Helps gain deeper insights into financial time series data

Financial time series smoothing

• Financial time series data often exhibit noise, irregularities, and fluctuations
• Smoothing techniques help reduce random variations and reveal underlying trends
• This presentation explores various smoothing methods used in financial time series analysis

Simple Moving Average (SMA)

• Basic smoothing technique that calculates the average of a fixed number of data points
• Formula: \(SMA(t) = \frac{y(t) + y(t-1) + ... + y(t-n+1)}{n}\)
• Widely used in technical analysis for identifying trends and generating trading signals
• Advantages: Easy to understand and implement, effective for removing high-frequency noise
• Limitations: Introduces a lag, sensitive to outliers, may distort underlying patterns

Exponential Moving Average (EMA)

• Assigns exponentially decreasing weights to older data points
• Formula: \(EMA(t) = \alpha \times y(t) + (1 - \alpha) \times EMA(t-1)\)
• Commonly used in technical analysis, forecasting, and signal processing
• Advantages: Responds quickly to changes, less lag than SMA, less sensitive to outliers
• Limitations: Requires tuning the smoothing parameter (\(\alpha\))

Weighted Moving Average (WMA)

• Assigns different weights to data points within the window
• Formula: \(WMA(t) = \frac{w_1 \times y(t) + w_2 \times y(t-1) + ... + w_n \times y(t-n+1)}{w_1 + w_2 + ... + w_n}\)
• Used in technical analysis, signal processing, and trend analysis
• Advantages: Allows for customized weighting schemes, can better capture underlying patterns
• Limitations: Requires careful selection of weights, inappropriate weights can distort the series

Savitzky-Golay Filter

• Performs local polynomial regression on a moving window of data points
• Widely used in signal processing, spectroscopy, and financial time series analysis
• Advantages: Preserves data features (peaks and valleys), handles noisy data effectively
• Limitations: Computationally expensive, choice of polynomial order and window size affects results

Lowess (Locally Weighted Scatterplot Smoothing)

• Non-parametric regression technique that fits a low-degree polynomial to localized subsets of data
• Useful for identifying non-linear trends and patterns in financial time series data
• Advantages: Effective for handling non-linear relationships, robust to outliers, captures complex patterns
• Limitations: Computationally intensive, sensitive to the choice of smoothing parameters

Kalman Filter

• Recursive algorithm that estimates the actual state of a dynamic system from noisy observations
• Widely used in finance for portfolio optimization, risk management, and forecasting
• Advantages: Optimal for linear systems with Gaussian noise, handles missing data, provides state estimates and uncertainties
• Limitations: Assumes linear system model and Gaussian noise, performance degrades for non-linear or non-Gaussian systems

Quantile-Quantile Plot Creation

# Create Q-Q plot to compare returns distribution to normal distribution
glen_m %>%
    ggplot(aes(sample = 100 * log_return)) +  # Multiply by 100 to show as percentage
    # Add Q-Q points
    stat_qq(
        distribution = stats::qnorm,  # Compare against normal distribution
        color = "blue",
        alpha = 0.6
    ) +
    # Add reference line
    stat_qq_line(
        distribution = stats::qnorm,
        colour = "red"
    ) +
    # Add labels and formatting
    labs(
        title = "Q-Q Plot of Glencore Stock Returns",
        subtitle = "Comparing returns distribution to normal distribution",
        x = "Theoretical Quantiles",
        y = "Sample Quantiles (%)"
    ) +
    theme_minimal()

Seasonal Plot Creation

# Create seasonal plot to visualize patterns within years
glen_m_ts %>% 
    ggseasonplot(
        year.labels = TRUE,          # Show labels for each year
        year.labels.left = TRUE,     # Place year labels on left side
        col = rainbow(10)            # Use different color for each year
    ) + 
    # Add labels and formatting
    labs(
        title = "Seasonal Plot: Glencore Returns",
        subtitle = "Monthly patterns across different years",
        x = "Month",
        y = "Returns"
    ) +
    theme_minimal() +
    # Adjust theme for better readability
    theme(
        legend.position = "right",
        axis.text.x = element_text(angle = 45, hjust = 1)
    )

Understanding Autocorrelation

• Autocorrelation measures how related a time series is with itself at different time lags
• Think of it like comparing today’s price with:
  • Yesterday’s price (lag 1)
  • Last week’s price (lag 5)
  • Last month’s price (lag 20)
• Values range from -1 to 1:
  • Close to 1: Strong positive relationship
  • Close to -1: Strong negative relationship
  • Close to 0: No relationship

What is White Noise?

• White noise is a sequence of random values with:
  • Constant mean (usually 0)
  • Constant variance
  • No correlation between values at different times
• Important because:
  • It’s the simplest type of random process
  • Used to model “pure randomness” in financial markets
  • Helps identify when a pattern is truly significant

Introduction to Kalman Filtering

• Named after Rudolf Kálmán
• Think of it like a GPS system:
  • Combines noisy measurements with predictions
  • Updates estimates as new data arrives
  • Gets better over time
• In finance, used for:
  • Estimating true price trends
  • Filtering out market noise
  • State space modeling

Understanding Q-Q Plots

• Q-Q (Quantile-Quantile) plots help assess if data follows a normal distribution
• How to read them:
  • Points following diagonal line → data is normally distributed
  • Points above line → heavier tails than normal
  • Points below line → lighter tails than normal
• Why important in finance:
  • Many models assume normal distribution
  • Helps identify potential risk in extreme events
  • Guides choice of statistical methods

Seasonal Patterns in Financial Data

• Many financial time series show recurring patterns
• Common types:
  • Daily patterns (market open/close)
  • Weekly patterns (weekend effect)
  • Monthly patterns (end-of-month effect)
  • Yearly patterns (tax-loss selling)
• Understanding seasonality helps:
  • Better timing of trades
  • Risk management
  • Performance attribution