ts.model_service package

Submodules

ts.model_service.model_service module

ModelService defines an API for base model service.

class ts.model_service.model_service.ModelService(model_name, model_dir, manifest, gpu=None)

Bases: object

ModelService wraps up all preprocessing, inference and postprocessing functions used by model service. It is defined in a flexible manner to be easily extended to support different frameworks.

handle(data, context)

Backward compatible handle function.

Parameters:

• data –

• context –

Returns:

abstract inference(data)

Wrapper function to run pre-process, inference and post-process functions.

Parameters:

data (list of object) – Raw input from request.

Returns:

data to be sent back

Return type:

list of outputs to be sent back to client.

initialize(context)

Internal initialize ModelService.

Parameters:

context – MMS context object

Returns:

abstract ping()

Ping to get system’s health.

Returns:

A message, “health”: “healthy!”, to show system is healthy.

Return type:

String

signature()

Signature for model service.

Returns:

Model service signature.

Return type:

Dict

class ts.model_service.model_service.SingleNodeService(model_name, model_dir, manifest, gpu=None)

Bases: ModelService

SingleNodeModel defines abstraction for model service which loads a single model.

inference(data)

Wrapper function to run preprocess, inference and postprocess functions.

Parameters:

data (list of object) – Raw input from request.

Returns:

data to be sent back

Return type:

list of outputs to be sent back to client.

Module contents

Model services code