Serve¶
These components aim to make it easier to interact with inference and serving tools such as torchserve.
- torchx.components.serve.torchserve(model_path: str, management_api: str, image: str = 'ghcr.io/pytorch/torchx:0.1.2dev0', params: Optional[Dict[str, object]] = None, dryrun: bool = False) → torchx.specs.api.AppDef[source]¶
Deploys the provided model to the given torchserve management API endpoint.
>>> from torchx.components.serve import torchserve >>> torchserve( ... model_path="s3://your-bucket/your-model.pt", ... management_api="http://torchserve:8081", ... ) AppDef(name='torchx-torchserve', ...)
- Parameters
model_path – The fsspec path to the model archive file.
management_api – The URL to the root of the torchserve management API.
image – Container to use.
params – torchserve parameters. See https://pytorch.org/serve/management_api.html#register-a-model
dryrun – Start the app, but does not perform actual work
- Returns
the TorchX application definition
- Return type