qualia_core.evaluation.host.Qualia module

class qualia_core.evaluation.host.Qualia.Result(metrics: 'dict[str, float]', time: 'float')

Bases: object

metrics: dict[str, float]

time: float

class qualia_core.evaluation.host.Qualia.Qualia(chunks: int | None = None)

Bases: Evaluator

Custom evaluation loop for Qualia host implementations like Qualia-CodeGen Linux example.

evaluate(framework: LearningFramework[Any], model_kind: str, dataset: RawDataModel, target: str, tag: str, limit: int | None = None, dataaugmentations: list[DataAugmentation] | None = None) → Stats