twinlab.Emulator.benchmark

Emulator.benchmark(params=<twinlab.params.BenchmarkParams object>, verbose=False)

Benchmark the performance of a trained emulator with a calibration curve.

A test dataset must have been defined in order for this to produce a meaningful result. This means that train_test_ratio must be less than 1 when training the emulator. The calibration curve can be plotted to show how well the training data fits to the emulator, and is calculated differently depending on the params chosen. The returned dataframe contains 100 rows for each output column of the emulator. These can be plotted to ascertain the performance of the emulator.

Parameters:

• params (BenchmarkParams, optional) – A parameter-configuration object that contains optional parameters for benchmarking an emulator.

• verbose (bool, optional) – Display detailed information about the operation while running.

Returns:

Either a pandas.DataFrame containing the calibration curve for an emulator, or None if there is no test data.

Return type:

pandas.DataFrame, None

Example

emulator = tl.Emulator("quickstart")
emulator.benchmark()

      y
0   0.0
1   0.0
2   0.0
3   0.0
4   0.0
..  ...
95  1.0
96  1.0
97  1.0
98  1.0
99  1.0