Testing

This section is dedicated to explain how the testing phase is managed inside cyhole library.

By assuming that the reader has downloaded locally the library (see Latest Library Version for more details). It is required to download also the python testing libraries:

• activate the python environment you want to use
• navigate to the cyhole/tests folder
• run the command pip install -r requirements.txt

This command is required to download pytest and its extensions used by cyhole for test management.

Configuration File

The cyhole library interacts with many external APIs that sometimes require an API Key in order to extract data. Since the usage of these API Keys could be subjected to costs, a proper configuration for the tests is required.

The configuration is managed by a configuration file called test.ini, which is loaded during the execution of the tests. By using this file will be possible to:

• manage externally API Keys
• enable/disable mock responses

Check the default test.default.ini file inside the repository for a complete list of the available configurations.

During the tests' execution the configuration is loaded into TestConfiguration class.

Mock Responses

During the implementation phase, could be not necessary to execute a call to the external API service, especially if the external system requires an API Key with a computational costs. For this reason, it was implementated a method to mock the external API responses and overcome this situation.

The usage of the mocker is defined inside the configuration file in the global/local variables called mock_response.

Note

Global mock_response value overwrites the local mock_response value.

If the mocker response functionality is enabled, then the tests will try to load a JSON file with the response schema stored from a previous call of the API. The root of the mocker files is defined inside the mock_folder global variable, and specificed for each Interaction inside the local mock_folder variable.

Every time the tests are executed with the mock response functionality disabled, the process automatically stores the response in a JSON file into the proper resources folder. To avoid the overwrite of the JSON file use the global mock_file_overwrite variable.

All the functionalities to load and store the mock responses are managed by the MockerManager class.

Example

The code below provides a test extracted from test_birdeye.py to explain the mocker usage:

test_birdeye.py

from pathlib import Path

import pytest
from pytest_mock import MockerFixture

from cyhole.birdeye import Birdeye
from cyhole.birdeye.schema import GetTokenListResponse

# load test config
from .config import load_config, MockerManager
config = load_config()

# create resources folder
mock_path = Path(config.mock_folder) / config.birdeye.mock_folder

class TestBirdeyePublic:
    """
        Class grouping all unit test associate to **PUBLIC** endpoints.
    """
    birdeye = Birdeye(api_key = config.birdeye.api_key)
    mocker = MockerManager(mock_path)

    def test_get_token_list_sync(self, mocker: MockerFixture) -> None:
        """
            Unit Test used to check the response schema of endpoint "Token - List" 
            for synchronous logic.

            Mock Response File: get_token_list.json
        """

        # load mock response
        mock_file_name = "get_token_list"
        if config.mock_response or config.birdeye.mock_response_public:
            mock_response = self.mocker.load_mock_response(mock_file_name, GetTokenListResponse)
            mocker.patch("cyhole.core.client.APIClient.api", return_value = mock_response)

        # execute request
        response = self.birdeye.client.get_token_list(limit = 1)

        # actual test
        assert isinstance(response, GetTokenListResponse)

        # store request (only not mock)
        if config.mock_file_overwrite and not config.birdeye.mock_response_public:
            self.mocker.store_mock_model(mock_file_name, response)

By analysing the code into different blocks, we first import all the required dependencies:

from pathlib import Path

import pytest
from pytest_mock import MockerFixture

from cyhole.birdeye import Birdeye
from cyhole.birdeye.schema import GetTokenListResponse

...

To hendle the mocker functionalities, we need to import the MockerManager class from pytest_mock library extension of pytest. We also import the Birdeye class and the GetTokenListResponse schema from the cyhole.birdeye module to use them in the test.

As next step, we load the configuration file by importing the load_config function and the MockerManager class:

...
from .config import load_config, MockerManager
config = load_config()

# create resources folder
mock_path = Path(config.mock_folder) / config.birdeye.mock_folder
...

For cyhole.birdeye extension, we decided to manage the tests in two different classes: TestBirdeyePublic and TestBirdeyePrivate. The first one is used to test the public endpoints, while the second one is used to test the private endpoints. In this case, we are testing the public endpoints, so we are using the TestBirdeyePublic class.

...

class TestBirdeyePublic:
    """
        Class grouping all unit test associate to **PUBLIC** endpoints.
    """
    birdeye = Birdeye(api_key = config.birdeye.api_key)
    mocker = MockerManager(mock_path)

...

The TestBirdeyePublic class contains the Birdeye object and the MockerManager that is used to load and store the mock responses. The mock_path variable is used to define the path where the mock responses are stored.

As a final step, we define the test method:

...

def test_get_token_list_sync(self, mocker: MockerFixture) -> None:
    """
        Unit Test used to check the response schema of endpoint "Token - List" 
        for synchronous logic.

        Mock Response File: get_token_list.json
    """

    # load mock response
    mock_file_name = "get_token_list"
    if config.mock_response or config.birdeye.mock_response_public:
        mock_response = self.mocker.load_mock_response(mock_file_name, GetTokenListResponse)
        mocker.patch("cyhole.core.client.APIClient.api", return_value = mock_response)

    # execute request
    response = self.birdeye.client.get_token_list(limit = 1)

    # actual test
    assert isinstance(response, GetTokenListResponse)

    # store request (only not mock)
    if config.mock_file_overwrite and not config.birdeye.mock_response_public:
        self.mocker.store_mock_model(mock_file_name, response)

Observe that the test method is defined with the mocker fixture, which is used to patch the APIClient.api method. If the mocker response functionality is enabled, the method load_mock_response is used to load the mock response from the JSON file. The patch method is used to mock the APIClient.api method and return the mock response.

The get_token_list method is executed, and the response is stored in the response variable. The test checks if the response is an instance of the GetTokenListResponse schema.

Finally, observe that if the mock_file_overwrite variable is enabled and the mock_response_public variable is disabled, the method store_mock_model is used to store the response in the JSON file.

sync and async

The cyhole library is designed to work with both synchronous and asynchronous logic. For this reason, every tests in charge to perform a call to an external API service should be implemented in both ways. To hendle async tests, the pytest library provides the pytest-asyncio extension that allows to run async tests in a synchronous environment.

The code below provides an example of how to was implemented the test for the get_token_list endpoint in both sync and async logic in cyhole.birdeye.

def test_get_token_list_sync(self, mocker: MockerFixture) -> None:
    """
        Unit Test used to check the response schema of endpoint "Token - List" 
        for synchronous logic.

        Mock Response File: get_token_list.json
    """

    # load mock response
    mock_file_name = "get_token_list"
    if config.mock_response or config.birdeye.mock_response_public:
        mock_response = self.mocker.load_mock_response(mock_file_name, GetTokenListResponse)
        mocker.patch("cyhole.core.client.APIClient.api", return_value = mock_response)

    # execute request
    response = self.birdeye.client.get_token_list(limit = 1)

    # actual test
    assert isinstance(response, GetTokenListResponse)

    # store request (only not mock)
    if config.mock_file_overwrite and not config.birdeye.mock_response_public:
        self.mocker.store_mock_model(mock_file_name, response)

@pytest.mark.asyncio
async def test_get_token_list_async(self, mocker: MockerFixture) -> None:
    """
        Unit Test used to check the response schema of endpoint "Token - List" 
        for asynchronous logic.

        Mock Response File: get_token_list.json
    """

    # load mock response
    mock_file_name = "get_token_list"
    if config.mock_response or config.birdeye.mock_response_public:
        mock_response = self.mocker.load_mock_response(mock_file_name, GetTokenListResponse)
        mocker.patch("cyhole.core.client.AsyncAPIClient.api", return_value = mock_response)

    # execute request
    async with self.birdeye.async_client as client:
        response = await client.get_token_list(limit = 1)

    # actual test
    assert isinstance(response, GetTokenListResponse)

Annex

config.py

config

SolscanConfiguration

Bases: BaseModel

Model in charge to manage the Solscan APIs.

mock_response class-attribute instance-attribute

mock_response: bool = True

Flag to enable/disable the mock responses.

mock_folder class-attribute instance-attribute

mock_folder: str = 'solscan'

Folder where the mock responses are stored.

api_v1_key class-attribute instance-attribute

api_v1_key: str | None = None

API key to access the Solscan V1 APIs.

api_v2_key class-attribute instance-attribute

api_v2_key: str | None = None

API key to access the Solscan V2 APIs.

SolanaFMConfiguration

Bases: BaseModel

Model in charge to manage the SolanaFM APIs.

mock_response class-attribute instance-attribute

mock_response: bool = True

Flag to enable/disable the mock responses.

mock_folder class-attribute instance-attribute

mock_folder: str = 'solana_fm'

Folder where the mock responses are stored.

api_key class-attribute instance-attribute

api_key: str | None = None

API key to access the SolanaFM APIs.

JupiterConfiguration

Bases: BaseModel

Model in charge to manage the Jupiter APIs.

mock_response class-attribute instance-attribute

mock_response: bool = True

Flag to enable/disable the mock responses.

mock_folder class-attribute instance-attribute

mock_folder: str = 'jupiter'

Folder where the mock responses are stored.

BirdeyeConfiguration

Bases: BaseModel

Model in charge to manage the birdeye APIs.

mock_response_public class-attribute instance-attribute

mock_response_public: bool = True

Flag to enable/disable the mock responses for the public APIs.

mock_response_private class-attribute instance-attribute

mock_response_private: bool = True

Flag to enable/disable the mock responses for the private APIs.

mock_folder class-attribute instance-attribute

mock_folder: str = 'birdeye'

Folder where the mock responses are stored.

api_key class-attribute instance-attribute

api_key: str = ''

API key to access the birdeye APIs.

TestConfiguration

Bases: BaseModel

Model in charge to manage the tests' configuration.

mock_response class-attribute instance-attribute

mock_response: bool = True

Flag to enable/disable the mock responses.

mock_folder class-attribute instance-attribute

mock_folder: str = 'tests/resources/mock'

Folder where the mock responses are stored.

mock_file_overwrite class-attribute instance-attribute

mock_file_overwrite: bool = False

Flag to enable/disable the overwrite of the mock files.

birdeye class-attribute instance-attribute

birdeye: BirdeyeConfiguration = BirdeyeConfiguration()

Birdeye configuration.

jupiter class-attribute instance-attribute

jupiter: JupiterConfiguration = JupiterConfiguration()

Jupiter configuration.

solana_fm class-attribute instance-attribute

solana_fm: SolanaFMConfiguration = SolanaFMConfiguration()

SolanaFM configuration.

solscan class-attribute instance-attribute

solscan: SolscanConfiguration = SolscanConfiguration()

Solscan configuration.

MockerManager

MockerManager(mock_path: Path)

Class used to manage the mock responses.

Source code in tests/config.py

def __init__(self, mock_path: Path) -> None:
    # set/create resources folder
    self.mock_path = mock_path
    mock_path.mkdir(parents = True, exist_ok = True)

load_mock_model

load_mock_model(
    file_name: str, response_model: Type[ResponseModel]
) -> ResponseModel

Use this function to load mock response model from a file.

Parameters:

Name	Type	Description	Default
file_name	str	file name containing the response to load. By assumption, the file must be a JSON file and the file name should not contain the extension.	required
response_model	Type[ResponseModel]	objects that inherit from pydantic.BaseModel and describe the response object element.	required

Source code in tests/config.py

def load_mock_model(self, file_name: str, response_model: Type[ResponseModel]) -> ResponseModel:
    """
        Use this function to load mock response model from a file.

        Parameters:
            file_name: file name containing the response to load.
                By assumption, the file must be a JSON file and the file name should not contain the extension.
            response_model: objects that inherit from `pydantic.BaseModel` and describe the response object element.
    """
    mock_file = f"{file_name}.json"
    mock_path_file = self.mock_path / mock_file

    if not mock_path_file.exists():
        raise Exception(f"mock file '{mock_file}' does not exist in '{self.mock_path}'.")

    with open(mock_path_file, "r") as file:
        data = json.loads(file.read())
        mock_response = response_model(**data)

    return mock_response

store_mock_model

store_mock_model(
    file_name: str, response: BaseModel
) -> None

Use this function to store the mock response model into a file.

Parameters:

Name	Type	Description	Default
file_name	str	file name containing the response to load. By assumption, the file must be a JSON file and the file name should not contain the extension.	required
response	BaseModel	objects that inherit from pydantic.BaseModel and describe the response object element.	required

Source code in tests/config.py

def store_mock_model(self, file_name: str, response: BaseModel) -> None:
    """
        Use this function to store the mock response model into a file.

        Parameters:
            file_name: file name containing the response to load.
                By assumption, the file must be a JSON file and the file name should not contain the extension.
            response: objects that inherit from `pydantic.BaseModel` and describe the response object element.
    """
    mock_file = f"{file_name}.json"
    mock_path_file = self.mock_path / mock_file

    with open(mock_path_file, "w") as file:
        file.write(response.model_dump_json(indent = 4, by_alias = True))

    return

load_mock_response

load_mock_response(
    file_name: str, response_model: Type[ResponseModel]
) -> Response

Use this function to load mock response model from a file and return a dummy Response object.

Parameters:

Name	Type	Description	Default
file_name	str	file name containing the response to load. By assumption, the file must be a JSON file and the file name should not contain the extension.	required
response_model	Type[ResponseModel]	objects that inherit from pydantic.BaseModel and describe the response object element.	required

Source code in tests/config.py

def load_mock_response(self, file_name: str, response_model: Type[ResponseModel]) -> Response:
    """
        Use this function to load mock response model from a file and return a dummy `Response` object.

        Parameters:
            file_name: file name containing the response to load.
                By assumption, the file must be a JSON file and the file name should not contain the extension.
            response_model: objects that inherit from `pydantic.BaseModel` and describe the response object element.
    """
    mock_model = self.load_mock_model(file_name, response_model)

    # create dummy response
    mock_response = Response()
    mock_response._content = mock_model.model_dump_json(by_alias = True, exclude_none = True).encode()
    mock_response.status_code = 200
    mock_response.encoding = "utf-8"

    return mock_response

adjust_content_json

adjust_content_json(content: str) -> bytes

Use this function to adjust an input string to ensure that can be used as content for a Response object and paresed as a JSON object.

Parameters:

Name	Type	Description	Default
content	str	string to adjust.	required

Source code in tests/config.py

def adjust_content_json(self, content: str) -> bytes:
    """
        Use this function to adjust an input string to ensure that 
        can be used as content for a `Response` object and paresed 
        as a JSON object.

        Parameters:
            content: string to adjust.
    """
    # adjustemts
    content = content.replace("'", '"')
    content = content.replace("False", "false")
    content = content.replace("True", "true")
    content = content.replace("None", "null")

    return content.encode()

load_config

load_config(
    path: str = "tests", file: str = "test.ini"
) -> TestConfiguration

This function is used to load the test configuration file.

It is strongly suggested to leave the default path and file name for the test configuration file. However, if a dedicated file is used, then is possible to load another file by providing the necessary information in the input variables.

Parameters:

Name	Type	Description	Default
path	str	path of the configuration file.	'tests'
file	str	name of the configuration file (with extension).	'test.ini'

Returns:

Type	Description
TestConfiguration	pydantic model of the test's configuration.

Source code in tests/config.py

def load_config(path: str = "tests", file: str = "test.ini") -> TestConfiguration:
    """
        This function is used to load the test configuration file.

        It is **strongly suggested** to leave the default `path` and `file` 
        name for the test configuration file. However, if a dedicated file 
        is used, then is possible to load another file by providing the 
        necessary information in the input variables.

        Parameters:
            path: path of the configuration file.
            file: name of the configuration file (with extension).

        Returns:
            pydantic model of the test's configuration.
    """
    config_path_file = f"{path}/{file}"
    config = configparser.ConfigParser()

    # load config
    config.read(config_path_file)
    test_config = TestConfiguration()

    # global
    test_config.mock_response = config.getboolean("global", "mock_response", fallback = test_config.mock_response)
    test_config.mock_folder = config.get("global", "mock_folder", fallback = test_config.mock_folder)
    test_config.mock_file_overwrite = config.getboolean("global", "mock_file_overwrite", fallback = test_config.mock_file_overwrite)

    # birdeye
    test_config.birdeye.mock_response_public = config.getboolean("birdeye", "mock_response_public", fallback = test_config.birdeye.mock_response_public)
    test_config.birdeye.mock_response_private = config.getboolean("birdeye", "mock_response_private", fallback = test_config.birdeye.mock_response_private)
    test_config.birdeye.mock_folder = config.get("birdeye", "mock_folder", fallback = test_config.birdeye.mock_folder)
    test_config.birdeye.api_key = config.get("birdeye", "api_key", fallback = test_config.birdeye.api_key)

    # jupiter
    test_config.jupiter.mock_response = config.getboolean("jupiter", "mock_response", fallback = test_config.jupiter.mock_response)
    test_config.jupiter.mock_folder = config.get("jupiter", "mock_folder", fallback = test_config.jupiter.mock_folder)

    # solana_fm
    test_config.solana_fm.mock_response = config.getboolean("solana_fm", "mock_response", fallback = test_config.solana_fm.mock_response)
    test_config.solana_fm.mock_folder = config.get("solana_fm", "mock_folder", fallback = test_config.solana_fm.mock_folder)
    test_config.solana_fm.api_key = config.get("solana_fm", "api_key", fallback = test_config.solana_fm.api_key)

    # solscan
    test_config.solscan.mock_response = config.getboolean("solscan", "mock_response", fallback = test_config.solscan.mock_response)
    test_config.solscan.mock_folder = config.get("solscan", "mock_folder", fallback = test_config.solscan.mock_folder)
    test_config.solscan.api_v1_key = config.get("solscan", "api_v1_key", fallback = test_config.solscan.api_v1_key)
    test_config.solscan.api_v2_key = config.get("solscan", "api_v2_key", fallback = test_config.solscan.api_v2_key)

    return test_config