Ensemble Modelling¶

In this Project, you are going to implement soft voting classifier machine learning model on Apple stock data. You can use other stock's data as well.

The notebook is structured as follows:

  1. Import Libraries and Data
  2. Split Data into Train and Test and Scale the Features
  3. Implement Voting Classifier
  4. Strategy Analysis

Import Libraries and Data¶

You will use the pandas and numpy libraries for data storage and manipulation. You will use the XGBoost, Logistic Regression, ADABoost, and SVM models which will be used by the voting classifier to predict the final output. You will be using the sklearn library for using the different ML models.

In [13]:
# Import libraries for data manipulation
import numpy as np
import pandas as pd
import os

# For machine learning models
import xgboost
from xgboost import XGBClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import VotingClassifier
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.ensemble import AdaBoostClassifier
from sklearn.linear_model import LogisticRegression
from sklearn import svm
from sklearn.preprocessing import StandardScaler

# Import matplotlib as an alias plt and set the style
import matplotlib.pyplot as plt
%matplotlib inline
plt.style.use("seaborn-v0_8-whitegrid")

# Import sys to append the path for custom function file
import sys
sys.path.append(os.path.abspath("../.."))

# Import the custom function file
from Support.advanced_momentum_1_1 import target_var, train_test_split, \
                                             generate_trade_sheet, trade_analytics, \
                                             performance_metrics, predict_signals
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')

Read the Data¶

You will use the Apple stock data stored in the file aapl_jan_2004_feb_2024.csv and aapl_jun_2020_jan_2024.csv.

Further, you will use the features generated from the encoder-decoder model stored in the csv file encoded_features_apple_momentum_jan_2006_jan_2024.csv. This file was created similar to the manner we created the features in the notebook Encoder-Decoder Approach to Reduce Features.

All the data files are stored in a downloadable zip file which is located in the summary section of the course.

In [ ]:
In [14]:
# Import the pandas library and read the CSV file containing stock data
stock_data = pd.read_csv(
    '../../Data/aapl_jan_2004_feb_2024.csv', index_col=0)

# Convert the index of the stock_data DataFrame to datetime format
stock_data.index = pd.to_datetime(stock_data.index)

aapl_test_prices = pd.read_csv(
    '../../Data/aapl_jun_2020_jan_2024.csv', index_col=0)

# Import the pandas library again and read the CSV file containing encoded features
features = pd.read_csv(
    '../../Data/encoded_features_apple_momentum_jan_2006_jan_2024.csv', index_col=0)

Calculate Target Variable¶

Now, you will create the target variable which checks whether the momentum indicator for a period of 20 is positive or not.

In [15]:
# Function call
target = target_var(stock_data, 20)

# Display the last 5 rows of the target dataframe
target.head()
Out[15]:
signal
Date
2004-01-02 1
2004-01-05 1
2004-01-06 -1
2004-01-07 -1
2004-01-08 -1

The features dataframe contains encoded features that start two years after the date range covered by the target dataframe. This is because certain technical indicators have a lookback period of 504. Thus, we will make sure both features and target variables are of the same length.

In [16]:
# Find the start date from the 'features' dataframe
start_date = features.index[0]

# Select rows from 'stock_data' starting from the start date
target = target.loc[start_date:]

Split Data into Train and Test and Scale the Features¶

You will use the function train_test_split saved in the amtmls_util_quantra file to perform the functions of splitting the dataset and scaling the X_test dataframe.

In [17]:
# Split dataset and store the scaler
X_train, X_test, y_train, y_test, scaler = train_test_split(features, target, split_proportion=0.8)

Implement Voting Classifier¶

You are using four machine learning models, XGBoost, Logistic Regression, ADABoost, and SVM models whose predicted output will be aggregated by the voting classifier model, and depending on the type of voting, the final output will be generated.

In [18]:
# --------------- Classifier 1: XGBoost ------------------------------------------
xgb = XGBClassifier(n_estimators=15, max_depth=3,
                    random_state=42, eval_metric='logloss')

# --------------- Classifier 2: Logistic Regression Classifier------------------------------------------
lr = LogisticRegression(random_state=42)

# --------------- Classifier 3: AdaBoost Classifier ------------------------------------------
ada = AdaBoostClassifier(n_estimators=15, random_state=42)

# --------------- Classifier 4: SVM------------------------------------------
svc = svm.SVC(kernel='rbf', probability=True, random_state=42)

# Define a list to store the different models
estimator = []
estimator.append(('LR', lr))
estimator.append(('XGB', xgb))
estimator.append(('ada', ada))
estimator.append(('SVC', svc))

Soft Voting Classifier Model¶

After defining the base learners, you will define the voting classifier model. For this capstone project, you will implement the soft voting classifier model now.

In [19]:
# Implement voting classifier with hard voting
vot_soft = VotingClassifier(estimators=estimator, voting='soft')

# Fit the voting classifier model
vot_soft.fit(X_train, y_train)
Out[19]:
VotingClassifier(estimators=[('LR', LogisticRegression(random_state=42)),
                             ('XGB',
                              XGBClassifier(base_score=None, booster=None,
                                            callbacks=None,
                                            colsample_bylevel=None,
                                            colsample_bynode=None,
                                            colsample_bytree=None, device=None,
                                            early_stopping_rounds=None,
                                            enable_categorical=False,
                                            eval_metric='logloss',
                                            feature_types=None,
                                            feature_weights=None, gamma=None,
                                            grow_poli...
                                            max_cat_threshold=None,
                                            max_cat_to_onehot=None,
                                            max_delta_step=None, max_depth=3,
                                            max_leaves=None,
                                            min_child_weight=None, missing=nan,
                                            monotone_constraints=None,
                                            multi_strategy=None,
                                            n_estimators=15, n_jobs=None,
                                            num_parallel_tree=None, ...)),
                             ('ada',
                              AdaBoostClassifier(n_estimators=15,
                                                 random_state=42)),
                             ('SVC', SVC(probability=True, random_state=42))],
                 voting='soft')
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Parameters
estimators estimators: list of (str, estimator) tuples

Invoking the ``fit`` method on the ``VotingClassifier`` will fit clones
of those original estimators that will be stored in the class attribute
``self.estimators_``. An estimator can be set to ``'drop'`` using
:meth:`set_params`.

.. versionchanged:: 0.21
``'drop'`` is accepted. Using None was deprecated in 0.22 and
support was removed in 0.24. 		[('LR', ...), ('XGB', ...), ...]
voting voting: {'hard', 'soft'}, default='hard'

If 'hard', uses predicted class labels for majority rule voting.
Else if 'soft', predicts the class label based on the argmax of
the sums of the predicted probabilities, which is recommended for
an ensemble of well-calibrated classifiers. 		'soft'
weights weights: array-like of shape (n_classifiers,), default=None

Sequence of weights (`float` or `int`) to weight the occurrences of
predicted class labels (`hard` voting) or class probabilities
before averaging (`soft` voting). Uses uniform weights if `None`. 		None
n_jobs n_jobs: int, default=None

The number of jobs to run in parallel for ``fit``.
``None`` means 1 unless in a :obj:`joblib.parallel_backend` context.
``-1`` means using all processors. See :term:`Glossary `
for more details.

.. versionadded:: 0.18 		None
flatten_transform flatten_transform: bool, default=True

Affects shape of transform output only when voting='soft'
If voting='soft' and flatten_transform=True, transform method returns
matrix with shape (n_samples, n_classifiers * n_classes). If
flatten_transform=False, it returns
(n_classifiers, n_samples, n_classes). 		True
verbose verbose: bool, default=False

If True, the time elapsed while fitting will be printed as it
is completed.

.. versionadded:: 0.23 		False
Parameters
penalty penalty: {'l1', 'l2', 'elasticnet', None}, default='l2'

Specify the norm of the penalty:

- `None`: no penalty is added;
- `'l2'`: add a L2 penalty term and it is the default choice;
- `'l1'`: add a L1 penalty term;
- `'elasticnet'`: both L1 and L2 penalty terms are added.

.. warning::
Some penalties may not work with some solvers. See the parameter
`solver` below, to know the compatibility between the penalty and
solver.

.. versionadded:: 0.19
l1 penalty with SAGA solver (allowing 'multinomial' + L1)

.. deprecated:: 1.8
`penalty` was deprecated in version 1.8 and will be removed in 1.10.
Use `l1_ratio` instead. `l1_ratio=0` for `penalty='l2'`, `l1_ratio=1` for
`penalty='l1'` and `l1_ratio` set to any float between 0 and 1 for
`'penalty='elasticnet'`. 		'deprecated'
C C: float, default=1.0

Inverse of regularization strength; must be a positive float.
Like in support vector machines, smaller values specify stronger
regularization. `C=np.inf` results in unpenalized logistic regression.
For a visual example on the effect of tuning the `C` parameter
with an L1 penalty, see:
:ref:`sphx_glr_auto_examples_linear_model_plot_logistic_path.py`. 		1.0
l1_ratio l1_ratio: float, default=0.0

The Elastic-Net mixing parameter, with `0 <= l1_ratio <= 1`. Setting
`l1_ratio=1` gives a pure L1-penalty, setting `l1_ratio=0` a pure L2-penalty.
Any value between 0 and 1 gives an Elastic-Net penalty of the form
`l1_ratio * L1 + (1 - l1_ratio) * L2`.

.. warning::
Certain values of `l1_ratio`, i.e. some penalties, may not work with some
solvers. See the parameter `solver` below, to know the compatibility between
the penalty and solver.

.. versionchanged:: 1.8
Default value changed from None to 0.0.

.. deprecated:: 1.8
`None` is deprecated and will be removed in version 1.10. Always use
`l1_ratio` to specify the penalty type. 		0.0
dual dual: bool, default=False

Dual (constrained) or primal (regularized, see also
:ref:`this equation `) formulation. Dual formulation
is only implemented for l2 penalty with liblinear solver. Prefer `dual=False`
when n_samples > n_features. 		False
tol tol: float, default=1e-4

Tolerance for stopping criteria. 		0.0001
fit_intercept fit_intercept: bool, default=True

Specifies if a constant (a.k.a. bias or intercept) should be
added to the decision function. 		True
intercept_scaling intercept_scaling: float, default=1

Useful only when the solver `liblinear` is used
and `self.fit_intercept` is set to `True`. In this case, `x` becomes
`[x, self.intercept_scaling]`,
i.e. a "synthetic" feature with constant value equal to
`intercept_scaling` is appended to the instance vector.
The intercept becomes
``intercept_scaling * synthetic_feature_weight``.

.. note::
The synthetic feature weight is subject to L1 or L2
regularization as all other features.
To lessen the effect of regularization on synthetic feature weight
(and therefore on the intercept) `intercept_scaling` has to be increased. 		1
class_weight class_weight: dict or 'balanced', default=None

Weights associated with classes in the form ``{class_label: weight}``.
If not given, all classes are supposed to have weight one.

The "balanced" mode uses the values of y to automatically adjust
weights inversely proportional to class frequencies in the input data
as ``n_samples / (n_classes * np.bincount(y))``.

Note that these weights will be multiplied with sample_weight (passed
through the fit method) if sample_weight is specified.

.. versionadded:: 0.17
*class_weight='balanced'* 		None
random_state random_state: int, RandomState instance, default=None

Used when ``solver`` == 'sag', 'saga' or 'liblinear' to shuffle the
data. See :term:`Glossary ` for details. 		42
solver solver: {'lbfgs', 'liblinear', 'newton-cg', 'newton-cholesky', 'sag', 'saga'}, default='lbfgs'

Algorithm to use in the optimization problem. Default is 'lbfgs'.
To choose a solver, you might want to consider the following aspects:

- 'lbfgs' is a good default solver because it works reasonably well for a wide
class of problems.
- For :term:`multiclass` problems (`n_classes >= 3`), all solvers except
'liblinear' minimize the full multinomial loss, 'liblinear' will raise an
error.
- 'newton-cholesky' is a good choice for
`n_samples` >> `n_features * n_classes`, especially with one-hot encoded
categorical features with rare categories. Be aware that the memory usage
of this solver has a quadratic dependency on `n_features * n_classes`
because it explicitly computes the full Hessian matrix.
- For small datasets, 'liblinear' is a good choice, whereas 'sag'
and 'saga' are faster for large ones;
- 'liblinear' can only handle binary classification by default. To apply a
one-versus-rest scheme for the multiclass setting one can wrap it with the
:class:`~sklearn.multiclass.OneVsRestClassifier`.

.. warning::
The choice of the algorithm depends on the penalty chosen (`l1_ratio=0`
for L2-penalty, `l1_ratio=1` for L1-penalty and `0 < l1_ratio < 1` for
Elastic-Net) and on (multinomial) multiclass support:

================= ======================== ======================
solver l1_ratio multinomial multiclass
================= ======================== ======================
'lbfgs' l1_ratio=0 yes
'liblinear' l1_ratio=1 or l1_ratio=0 no
'newton-cg' l1_ratio=0 yes
'newton-cholesky' l1_ratio=0 yes
'sag' l1_ratio=0 yes
'saga' 0<=l1_ratio<=1 yes
================= ======================== ======================

.. note::
'sag' and 'saga' fast convergence is only guaranteed on features
with approximately the same scale. You can preprocess the data with
a scaler from :mod:`sklearn.preprocessing`.

.. seealso::
Refer to the :ref:`User Guide ` for more
information regarding :class:`LogisticRegression` and more specifically the
:ref:`Table `
summarizing solver/penalty supports.

.. versionadded:: 0.17
Stochastic Average Gradient (SAG) descent solver. Multinomial support in
version 0.18.
.. versionadded:: 0.19
SAGA solver.
.. versionchanged:: 0.22
The default solver changed from 'liblinear' to 'lbfgs' in 0.22.
.. versionadded:: 1.2
newton-cholesky solver. Multinomial support in version 1.6. 		'lbfgs'
max_iter max_iter: int, default=100

Maximum number of iterations taken for the solvers to converge. 		100
verbose verbose: int, default=0

For the liblinear and lbfgs solvers set verbose to any positive
number for verbosity. 		0
warm_start warm_start: bool, default=False

When set to True, reuse the solution of the previous call to fit as
initialization, otherwise, just erase the previous solution.
Useless for liblinear solver. See :term:`the Glossary `.

.. versionadded:: 0.17
*warm_start* to support *lbfgs*, *newton-cg*, *sag*, *saga* solvers. 		False
n_jobs n_jobs: int, default=None

Does not have any effect.

.. deprecated:: 1.8
`n_jobs` is deprecated in version 1.8 and will be removed in 1.10. 		None
Parameters
objective objective: typing.Union[str, xgboost.sklearn._SklObjWProto, typing.Callable[[typing.Any, typing.Any], typing.Tuple[numpy.ndarray, numpy.ndarray]], NoneType]

Specify the learning task and the corresponding learning objective or a custom
objective function to be used.

For custom objective, see :doc:`/tutorials/custom_metric_obj` and
:ref:`custom-obj-metric` for more information, along with the end note for
function signatures. 		'binary:logistic'
base_score base_score: typing.Union[float, typing.List[float], NoneType]

The initial prediction score of all instances, global bias. 		None
booster None
callbacks callbacks: typing.Optional[typing.List[xgboost.callback.TrainingCallback]]

List of callback functions that are applied at end of each iteration.
It is possible to use predefined callbacks by using
:ref:`Callback API `.

.. note::

States in callback are not preserved during training, which means callback
objects can not be reused for multiple training sessions without
reinitialization or deepcopy.

.. code-block:: python

for params in parameters_grid:
# be sure to (re)initialize the callbacks before each run
callbacks = [xgb.callback.LearningRateScheduler(custom_rates)]
reg = xgboost.XGBRegressor(**params, callbacks=callbacks)
reg.fit(X, y) 		None
colsample_bylevel colsample_bylevel: typing.Optional[float]

Subsample ratio of columns for each level. 		None
colsample_bynode colsample_bynode: typing.Optional[float]

Subsample ratio of columns for each split. 		None
colsample_bytree colsample_bytree: typing.Optional[float]

Subsample ratio of columns when constructing each tree. 		None
device device: typing.Optional[str]

.. versionadded:: 2.0.0

Device ordinal, available options are `cpu`, `cuda`, and `gpu`. 		None
early_stopping_rounds early_stopping_rounds: typing.Optional[int]

.. versionadded:: 1.6.0

- Activates early stopping. Validation metric needs to improve at least once in
every **early_stopping_rounds** round(s) to continue training. Requires at
least one item in **eval_set** in :py:meth:`fit`.

- If early stopping occurs, the model will have two additional attributes:
:py:attr:`best_score` and :py:attr:`best_iteration`. These are used by the
:py:meth:`predict` and :py:meth:`apply` methods to determine the optimal
number of trees during inference. If users want to access the full model
(including trees built after early stopping), they can specify the
`iteration_range` in these inference methods. In addition, other utilities
like model plotting can also use the entire model.

- If you prefer to discard the trees after `best_iteration`, consider using the
callback function :py:class:`xgboost.callback.EarlyStopping`.

- If there's more than one item in **eval_set**, the last entry will be used for
early stopping. If there's more than one metric in **eval_metric**, the last
metric will be used for early stopping. 		None
enable_categorical enable_categorical: bool

See the same parameter of :py:class:`DMatrix` for details. 		False
eval_metric eval_metric: typing.Union[str, typing.List[typing.Union[str, typing.Callable]], typing.Callable, NoneType]

.. versionadded:: 1.6.0

Metric used for monitoring the training result and early stopping. It can be a
string or list of strings as names of predefined metric in XGBoost (See
:doc:`/parameter`), one of the metrics in :py:mod:`sklearn.metrics`, or any
other user defined metric that looks like `sklearn.metrics`.

If custom objective is also provided, then custom metric should implement the
corresponding reverse link function.

Unlike the `scoring` parameter commonly used in scikit-learn, when a callable
object is provided, it's assumed to be a cost function and by default XGBoost
will minimize the result during early stopping.

For advanced usage on Early stopping like directly choosing to maximize instead
of minimize, see :py:obj:`xgboost.callback.EarlyStopping`.

See :doc:`/tutorials/custom_metric_obj` and :ref:`custom-obj-metric` for more
information.

.. code-block:: python

from sklearn.datasets import load_diabetes
from sklearn.metrics import mean_absolute_error
X, y = load_diabetes(return_X_y=True)
reg = xgb.XGBRegressor(
tree_method="hist",
eval_metric=mean_absolute_error,
)
reg.fit(X, y, eval_set=[(X, y)]) 		'logloss'
feature_types feature_types: typing.Optional[typing.Sequence[str]]

.. versionadded:: 1.7.0

Used for specifying feature types without constructing a dataframe. See
the :py:class:`DMatrix` for details. 		None
feature_weights feature_weights: Optional[ArrayLike]

Weight for each feature, defines the probability of each feature being selected
when colsample is being used. All values must be greater than 0, otherwise a
`ValueError` is thrown. 		None
gamma gamma: typing.Optional[float]

(min_split_loss) Minimum loss reduction required to make a further partition on
a leaf node of the tree. 		None
grow_policy grow_policy: typing.Optional[str]

Tree growing policy.

- depthwise: Favors splitting at nodes closest to the node,
- lossguide: Favors splitting at nodes with highest loss change. 		None
importance_type None
interaction_constraints interaction_constraints: typing.Union[str, typing.List[typing.Tuple[str]], NoneType]

Constraints for interaction representing permitted interactions. The
constraints must be specified in the form of a nested list, e.g. ``[[0, 1], [2,
3, 4]]``, where each inner list is a group of indices of features that are
allowed to interact with each other. See :doc:`tutorial
` for more information 		None
learning_rate learning_rate: typing.Optional[float]

Boosting learning rate (xgb's "eta") 		None
max_bin max_bin: typing.Optional[int]

If using histogram-based algorithm, maximum number of bins per feature 		None
max_cat_threshold max_cat_threshold: typing.Optional[int]

.. versionadded:: 1.7.0

.. note:: This parameter is experimental

Maximum number of categories considered for each split. Used only by
partition-based splits for preventing over-fitting. Also, `enable_categorical`
needs to be set to have categorical feature support. See :doc:`Categorical Data
` and :ref:`cat-param` for details. 		None
max_cat_to_onehot max_cat_to_onehot: Optional[int]

.. versionadded:: 1.6.0

.. note:: This parameter is experimental

A threshold for deciding whether XGBoost should use one-hot encoding based split
for categorical data. When number of categories is lesser than the threshold
then one-hot encoding is chosen, otherwise the categories will be partitioned
into children nodes. Also, `enable_categorical` needs to be set to have
categorical feature support. See :doc:`Categorical Data
` and :ref:`cat-param` for details. 		None
max_delta_step max_delta_step: typing.Optional[float]

Maximum delta step we allow each tree's weight estimation to be. 		None
max_depth max_depth: typing.Optional[int]

Maximum tree depth for base learners. 		3
max_leaves max_leaves: typing.Optional[int]

Maximum number of leaves; 0 indicates no limit. 		None
min_child_weight min_child_weight: typing.Optional[float]

Minimum sum of instance weight(hessian) needed in a child. 		None
missing missing: float

Value in the data which needs to be present as a missing value. Default to
:py:data:`numpy.nan`. 		nan
monotone_constraints monotone_constraints: typing.Union[typing.Dict[str, int], str, NoneType]

Constraint of variable monotonicity. See :doc:`tutorial `
for more information. 		None
multi_strategy multi_strategy: typing.Optional[str]

.. versionadded:: 2.0.0

.. note:: This parameter is working-in-progress.

The strategy used for training multi-target models, including multi-target
regression and multi-class classification. See :doc:`/tutorials/multioutput` for
more information.

- ``one_output_per_tree``: One model for each target.
- ``multi_output_tree``: Use multi-target trees. 		None
n_estimators n_estimators: Optional[int]

Number of boosting rounds. 		15
n_jobs n_jobs: typing.Optional[int]

Number of parallel threads used to run xgboost. When used with other
Scikit-Learn algorithms like grid search, you may choose which algorithm to
parallelize and balance the threads. Creating thread contention will
significantly slow down both algorithms. 		None
num_parallel_tree None
random_state random_state: typing.Union[numpy.random.mtrand.RandomState, numpy.random._generator.Generator, int, NoneType]

Random number seed.

.. note::

Using gblinear booster with shotgun updater is nondeterministic as
it uses Hogwild algorithm. 		42
reg_alpha reg_alpha: typing.Optional[float]

L1 regularization term on weights (xgb's alpha). 		None
reg_lambda reg_lambda: typing.Optional[float]

L2 regularization term on weights (xgb's lambda). 		None
sampling_method sampling_method: typing.Optional[str]

Sampling method. Used only by the GPU version of ``hist`` tree method.

- ``uniform``: Select random training instances uniformly.
- ``gradient_based``: Select random training instances with higher probability
when the gradient and hessian are larger. (cf. CatBoost) 		None
scale_pos_weight scale_pos_weight: typing.Optional[float]

Balancing of positive and negative weights. 		None
subsample subsample: typing.Optional[float]

Subsample ratio of the training instance. 		None
tree_method tree_method: typing.Optional[str]

Specify which tree method to use. Default to auto. If this parameter is set to
default, XGBoost will choose the most conservative option available. It's
recommended to study this option from the parameters document :doc:`tree method
` 		None
validate_parameters validate_parameters: typing.Optional[bool]

Give warnings for unknown parameter. 		None
verbosity verbosity: typing.Optional[int]

The degree of verbosity. Valid values are 0 (silent) - 3 (debug). 		None
Parameters
estimator estimator: object, default=None

The base estimator from which the boosted ensemble is built.
Support for sample weighting is required, as well as proper
``classes_`` and ``n_classes_`` attributes. If ``None``, then
the base estimator is :class:`~sklearn.tree.DecisionTreeClassifier`
initialized with `max_depth=1`.

.. versionadded:: 1.2
`base_estimator` was renamed to `estimator`. 		None
n_estimators n_estimators: int, default=50

The maximum number of estimators at which boosting is terminated.
In case of perfect fit, the learning procedure is stopped early.
Values must be in the range `[1, inf)`. 		15
learning_rate learning_rate: float, default=1.0

Weight applied to each classifier at each boosting iteration. A higher
learning rate increases the contribution of each classifier. There is
a trade-off between the `learning_rate` and `n_estimators` parameters.
Values must be in the range `(0.0, inf)`. 		1.0
random_state random_state: int, RandomState instance or None, default=None

Controls the random seed given at each `estimator` at each
boosting iteration.
Thus, it is only used when `estimator` exposes a `random_state`.
Pass an int for reproducible output across multiple function calls.
See :term:`Glossary `. 		42
Parameters
C C: float, default=1.0

Regularization parameter. The strength of the regularization is
inversely proportional to C. Must be strictly positive. The penalty
is a squared l2 penalty. For an intuitive visualization of the effects
of scaling the regularization parameter C, see
:ref:`sphx_glr_auto_examples_svm_plot_svm_scale_c.py`. 		1.0
kernel kernel: {'linear', 'poly', 'rbf', 'sigmoid', 'precomputed'} or callable, default='rbf'

Specifies the kernel type to be used in the algorithm. If
none is given, 'rbf' will be used. If a callable is given it is used to
pre-compute the kernel matrix from data matrices; that matrix should be
an array of shape ``(n_samples, n_samples)``. For an intuitive
visualization of different kernel types see
:ref:`sphx_glr_auto_examples_svm_plot_svm_kernels.py`. 		'rbf'
degree degree: int, default=3

Degree of the polynomial kernel function ('poly').
Must be non-negative. Ignored by all other kernels. 		3
gamma gamma: {'scale', 'auto'} or float, default='scale'

Kernel coefficient for 'rbf', 'poly' and 'sigmoid'.

- if ``gamma='scale'`` (default) is passed then it uses
1 / (n_features * X.var()) as value of gamma,
- if 'auto', uses 1 / n_features
- if float, must be non-negative.

.. versionchanged:: 0.22
The default value of ``gamma`` changed from 'auto' to 'scale'. 		'scale'
coef0 coef0: float, default=0.0

Independent term in kernel function.
It is only significant in 'poly' and 'sigmoid'. 		0.0
shrinking shrinking: bool, default=True

Whether to use the shrinking heuristic.
See the :ref:`User Guide `. 		True
probability probability: bool, default=False

Whether to enable probability estimates. This must be enabled prior
to calling `fit`, will slow down that method as it internally uses
5-fold cross-validation, and `predict_proba` may be inconsistent with
`predict`. Read more in the :ref:`User Guide `. 		True
tol tol: float, default=1e-3

Tolerance for stopping criterion. 		0.001
cache_size cache_size: float, default=200

Specify the size of the kernel cache (in MB). 		200
class_weight class_weight: dict or 'balanced', default=None

Set the parameter C of class i to class_weight[i]*C for
SVC. If not given, all classes are supposed to have
weight one.
The "balanced" mode uses the values of y to automatically adjust
weights inversely proportional to class frequencies in the input data
as ``n_samples / (n_classes * np.bincount(y))``. 		None
verbose verbose: bool, default=False

Enable verbose output. Note that this setting takes advantage of a
per-process runtime setting in libsvm that, if enabled, may not work
properly in a multithreaded context. 		False
max_iter max_iter: int, default=-1

Hard limit on iterations within solver, or -1 for no limit. 		-1
decision_function_shape decision_function_shape: {'ovo', 'ovr'}, default='ovr'

Whether to return a one-vs-rest ('ovr') decision function of shape
(n_samples, n_classes) as all other classifiers, or the original
one-vs-one ('ovo') decision function of libsvm which has shape
(n_samples, n_classes * (n_classes - 1) / 2). However, note that
internally, one-vs-one ('ovo') is always used as a multi-class strategy
to train models; an ovr matrix is only constructed from the ovo matrix.
The parameter is ignored for binary classification.

.. versionchanged:: 0.19
decision_function_shape is 'ovr' by default.

.. versionadded:: 0.17
*decision_function_shape='ovr'* is recommended.

.. versionchanged:: 0.17
Deprecated *decision_function_shape='ovo' and None*. 		'ovr'
break_ties break_ties: bool, default=False

If true, ``decision_function_shape='ovr'``, and number of classes > 2,
:term:`predict` will break ties according to the confidence values of
:term:`decision_function`; otherwise the first class among the tied
classes is returned. Please note that breaking ties comes at a
relatively high computational cost compared to a simple predict. See
:ref:`sphx_glr_auto_examples_svm_plot_svm_tie_breaking.py` for an
example of its usage with ``decision_function_shape='ovr'``.

.. versionadded:: 0.22 		False
random_state random_state: int, RandomState instance or None, default=None

Controls the pseudo random number generation for shuffling the data for
probability estimates. Ignored when `probability` is False.
Pass an int for reproducible output across multiple function calls.
See :term:`Glossary `. 		42

Use Trained Model to Predict Target Variable¶

Once you have trained the model, you will use the test features dataset which has not been seen by the model and give the signal buy, which is 1, or sell, which is -1. You will use the function predict_signals, which takes the test dataset and using the trained model, predicts the target variable and provides the signal.

In [20]:
# Generate trading signals
signals = predict_signals(X_test, aapl_test_prices, vot_soft, scaler)

Strategy Analysis¶

Now that you have predicted whether the stock is in momentum or not, you will generate the trade sheet.

In [21]:
# Generate trade sheet
trade_sheet = generate_trade_sheet(signals)

# Display trade sheet
trade_sheet
Out[21]:
Position Entry Date Entry Price Exit Date Exit Price PnL
0 1.0 08-06-2020 81.53 06-10-2020 110.86 29.33
1 -1.0 07-10-2020 112.74 08-04-2021 128.13 -15.39
2 1.0 09-04-2021 130.72 08-06-2021 124.78 -5.94
3 -1.0 09-06-2021 125.16 08-07-2021 141.02 -15.86
4 1.0 09-07-2021 142.87 06-08-2021 144.09 1.22
5 -1.0 09-08-2021 144.05 07-09-2021 154.50 -10.45
6 1.0 08-09-2021 152.94 06-10-2021 140.01 -12.93
7 -1.0 07-10-2021 141.28 04-11-2021 148.85 -7.57
8 1.0 05-11-2021 149.38 08-03-2022 155.66 6.28
9 -1.0 09-03-2022 161.11 06-04-2022 169.89 -8.78
10 1.0 07-04-2022 170.20 06-05-2022 155.73 -14.47
11 -1.0 09-05-2022 150.56 07-06-2022 147.25 3.31
12 1.0 08-06-2022 146.50 09-08-2023 177.49 30.99
13 -1.0 10-08-2023 177.27 09-10-2023 178.53 -1.26
14 1.0 10-10-2023 177.93 07-12-2023 194.02 16.09

You will use the function trade_analytics which gives the summary of the trades carried out.

In [22]:
# Trade Analytics
trade_analysis = trade_analytics(trade_sheet)

# Display the analytics
trade_analysis
Out[22]:
Strategy
Total PnL -5.43
Total Trades 15
Number of Winners 6
Number of Losers 9
Win (%) 40.0
Loss (%) 60.0
Average Profit of Winning Trade 14.54
Average Loss of Losing Trade 10.29
Average Holding Time 34 days 16:00:00
Profit Factor 0.94

You can see that from June 8, 2020, to December 7, 2023, the momentum trading strategy-based ML model took 15 trades with different profit and loss amounts. The strategy is overall positive with a profit of $32.3. The profit factor is 1.48 with 8 winning trades and 7 losing trades. Let's check the performance analytics now.

In [23]:
# Analyse performance metrics
performance = performance_metrics(signals)
No description has been provided for this image
No description has been provided for this image 
                      Strategy
CAGR                    -1.43%
Annualised Volatility   29.74%
Sharpe Ratio              0.03
Maximum Drawdown       -63.35%

Conclusion¶

You have tried the momentum trading strategy on Apple stock data using multiple ML models and the reduced features which were created by the autoencoder (Encoder-Decoder) model. You can tweak this strategy by using different features and combining different ML models as well.