ebonite package¶
-
ebonite.
load_extensions
(*exts)[source]¶ Load extensions
Parameters: exts – list of extension main modules
-
class
ebonite.
Ebonite
(meta_repo: ebonite.repository.metadata.base.MetadataRepository, artifact_repo: ebonite.repository.artifact.base.ArtifactRepository, dataset_repo: ebonite.repository.dataset.base.DatasetRepository = None)[source]¶ Bases:
object
Main entry point for ebonite
This is the client for Ebonite API. It can save, load and build Models, Tasks and Projects. Ebonite instance can be obtained from factory methods like
local()
for local client,inmemory()
for inmemory client.You can save client config with
save_client_config()
and later restore it withfrom_config_file()
Parameters: - meta_repo –
MetadataRepository
instance to save metadata - artifact_repo –
ArtifactRepository
instance to save artifacts
-
default_server
= None¶
-
default_env
= None¶
-
push_model
(model: ebonite.core.objects.core.Model, task: ebonite.core.objects.core.Task = None) → ebonite.core.objects.core.Model[source]¶ Pushes
Model
instance into metadata and artifact repositoriesParameters: task :return: same saved
Model
instance
-
create_model
(model_object, model_input, model_name: str = None, *, project_name: str = 'default_project', task_name: str = 'default_task', **kwargs)[source]¶ This function creates ebonite model. Creates model, task and project (if needed) and pushes it to repo
Parameters: - model_object – object containing model.
- model_input – model input.
- model_name – model name to create.
- project_name – project name.
- task_name – task name.
- kwargs – other arguments for model
Returns: Model
instance representing
-
get_model
(model_name: str, task: Union[int, str, core.Task], project: Union[int, str, core.Project] = None, load_artifacts: bool = True) → ebonite.core.objects.core.Model[source]¶ Load model from repository
Parameters: Returns: Model
instance
-
create_image
(obj, name: str = None, task: ebonite.core.objects.core.Task = None, server: ebonite.runtime.server.base.Server = None, environment: ebonite.core.objects.core.RuntimeEnvironment = None, debug=False, skip_build=False, builder_args: Dict[str, object] = None, **kwargs) → ebonite.core.objects.core.Image[source]¶ Builds image of model service and stores it to repository
Parameters: - obj – model/list of models/pipeline or any object that has existing Hook for it to wrap into service
- name – name of image to build
- task – task to put image into
- server – server to build image with
- environment – env to build for
- debug – flag to build debug image
- skip_build – wheter to skip actual image build
- builder_args – kwargs for image.build
- kwargs – additional kwargs for builder
Returns: Image
instance representing built image
-
create_instance
(image: ebonite.core.objects.core.Image, name: str = None, environment: ebonite.core.objects.core.RuntimeEnvironment = None, run=False, runner_kwargs: Dict[str, object] = None, **instance_kwargs) → ebonite.core.objects.core.RuntimeInstance[source]¶ Runs model service instance and stores it to repository
Parameters: - image – image to run instance from
- name – name of instance to run
- environment – environment to run instance in, if no given localhost is used
- run – whether to automatically run instance after creation
- runner_kwargs – additional parameters for runner
- instance_kwargs – additional parameters for instance
Returns: RuntimeInstance
instance representing run instance
-
build_and_run_instance
(obj, name: str = None, task: ebonite.core.objects.core.Task = None, environment: ebonite.core.objects.core.RuntimeEnvironment = None, builder_kwargs: Dict[str, object] = None, runner_kwargs: Dict[str, object] = None, instance_kwargs: Dict[str, object] = None) → ebonite.core.objects.core.RuntimeInstance[source]¶ Builds image of model service, immediately runs service and stores both image and instance to repository
Parameters: - obj – buildable object to wrap into service
- name – name of image and instance to be built and run respectively
- task – task to put image into
- environment – environment to run instance in, if no given localhost is used
- builder_kwargs – additional kwargs for builder
- runner_kwargs – additional parameters for runner. Full list can be seen in https://docker-py.readthedocs.io/en/stable/containers.html
- instance_kwargs – additional parameters for instance
Returns: RuntimeInstance
instance representing run instance
-
classmethod
local
(path=None, clear=False) → ebonite.client.base.Ebonite[source]¶ Get an instance of
Ebonite
that stores metadata and artifacts on local filesystemParameters: - path – path to storage dir. If None, .ebonite dir is used
- clear – if True, erase previous data from storage
-
classmethod
inmemory
() → ebonite.client.base.Ebonite[source]¶ Get an instance of
Ebonite
with inmemory repositories
-
classmethod
custom_client
(metadata: Union[str, ebonite.repository.metadata.base.MetadataRepository], artifact: Union[str, ebonite.repository.artifact.base.ArtifactRepository], meta_kwargs: dict = None, artifact_kwargs: dict = None) → ebonite.client.base.Ebonite[source]¶ Create custom Ebonite client from metadata and artifact repositories.
Parameters: - metadata –
MetadataRepository
instance or pyjackson subtype type name - artifact –
ArtifactRepository
instance or pyjackson subtype type name - meta_kwargs – kwargs for metadata repo __init__ if subtype type name was provided
- artifact_kwargs – kwargs for artifact repo __init__ if subtype type name was provided
Returns: Ebonite
instance- metadata –
-
classmethod
from_config_file
(filepath) → ebonite.client.base.Ebonite[source]¶ Read and create Ebonite instance from config file
Parameters: filepath – path to read config from Returns: Ebonite
instance
-
save_client_config
(filepath)[source]¶ Save current client config to a file
Parameters: filepath – path to file
-
get_default_environment
()[source]¶ Creates (if needed) and returns default runtime environment
Returns: saved instance of RuntimeEnvironment
-
push_environment
(environment: ebonite.core.objects.core.RuntimeEnvironment) → ebonite.core.objects.core.RuntimeEnvironment[source]¶ Creates runtime environment in the repository
Parameters: environment – runtime environment to create Returns: created runtime environment Exception: errors.ExistingEnvironmentError
if given runtime environment has the same name as existing
-
get_environment
(name: str) → Optional[ebonite.core.objects.core.RuntimeEnvironment][source]¶ Finds runtime environment by name.
Parameters: name – expected runtime environment name Returns: found runtime environment if exists or None
-
get_environments
() → List[ebonite.core.objects.core.RuntimeEnvironment][source]¶ Gets a list of runtime environments
Returns: found runtime environments
-
get_image
(image_name: str, task: Union[int, str, core.Task], project: Union[int, str, core.Project] = None) → Optional[Image][source]¶ Finds image by name in given model, task and project.
Parameters: - image_name – expected image name
- task – task to search for image in
- project – project to search for image in
Returns: found image if exists or None
-
get_images
(task: Union[int, str, core.Task], project: Union[int, str, core.Project] = None) → List[Image][source]¶ Gets a list of images in given model, task and project
Parameters: - task – task to search for images in
- project – project to search for images in
Returns: found images
-
get_instance
(instance_name: str, image: Union[int, Image], environment: Union[int, RuntimeEnvironment]) → Optional[ebonite.core.objects.core.RuntimeInstance][source]¶ Finds instance by name in given image and environment.
Parameters: - instance_name – expected instance name
- image – image (or id) to search for instance in
- environment – environment (or id) to search for instance in
Returns: found instance if exists or None
-
get_instances
(image: Union[int, Image] = None, environment: Union[int, RuntimeEnvironment] = None) → List[ebonite.core.objects.core.RuntimeInstance][source]¶ Gets a list of instances in given image or environment
Parameters: - image – image (or id) to search for instances in
- environment – environment (or id) to search for instances in
Returns: found instances
-
get_models
(task: Union[int, str, core.Task], project: Union[int, str, core.Project] = None) → List[Model][source]¶ Gets a list of models in given project and task
Parameters: - task – task to search for models in
- project – project to search for models in
Returns: found models
-
get_or_create_project
(name: str) → ebonite.core.objects.core.Project[source]¶ Creates a project if not exists or gets existing project otherwise.
Parameters: name – project name Returns: project
-
get_or_create_task
(project: str, task_name: str) → ebonite.core.objects.core.Task[source]¶ Creates a task if not exists or gets existing task otherwise.
Parameters: - project – project to search/create task in
- task_name – expected name of task
Returns: created/found task
-
get_pipeline
(pipeline_name: str, task: Union[int, str, core.Task], project: Union[int, str, core.Project] = None) → Optional[Pipeline][source]¶ Finds model by name in given task and project.
Parameters: - pipeline_name – expected pipeline name
- task – task to search for pipeline in
- project – project to search for pipeline in
Returns: found pipeline if exists or None
-
get_pipelines
(task: Union[int, str, core.Task], project: Union[int, str, core.Project] = None) → List[Pipeline][source]¶ Gets a list of pipelines in given project and task
Parameters: - task – task to search for models in
- project – project to search for models in
Returns: found pipelines
-
get_project
(name: str) → Optional[ebonite.core.objects.core.Project][source]¶ Finds project in the repository by name
Parameters: name – name of the project to return Returns: found project if exists or None
-
get_projects
() → List[ebonite.core.objects.core.Project][source]¶ Gets all projects in the repository
Returns: all projects in the repository
-
get_task
(project: Union[int, str, core.Project], task_name: str) → Optional[Task][source]¶ Finds task with given name in given project
Parameters: - project – project to search for task in
- task_name – expected name of task
Returns: task if exists or None
-
get_tasks
(project: Union[int, str, core.Project]) → List[Task][source]¶ Gets a list of tasks for given project
Parameters: project – project to search for tasks in Returns: project tasks
-
delete_project
(project: ebonite.core.objects.core.Project, cascade: bool = False)[source]¶ ” Deletes project and(if required) all tasks associated with it from metadata repository
Parameters: - project – project to delete
- cascade – whether should project be deleted with all associated tasks
Returns: Nothing
-
delete_task
(task: ebonite.core.objects.core.Task, cascade: bool = False)[source]¶ ” Deletes task from metadata
Parameters: - task – task to delete
- cascade – whether should task be deleted with all associated objects
Returns: Nothing
-
delete_model
(model: ebonite.core.objects.core.Model, force: bool = False)[source]¶ ” Deletes model from metadata and artifact repositories
Parameters: - model – model to delete
- force – whether model artifacts’ deletion errors should be ignored, default is false
Returns: Nothing
-
delete_pipeline
(pipeline: ebonite.core.objects.core.Pipeline)[source]¶ “Deletes pipeline from metadata
Parameters: pipeline – pipeline to delete
-
delete_image
(image: ebonite.core.objects.core.Image, meta_only: bool = False, cascade: bool = False)[source]¶ ” Deletes existing image from metadata repository and image provider
Parameters: - image – image ot delete
- meta_only – should image be deleted only from metadata
- cascade – whether to delete nested RuntimeInstances
-
delete_instance
(instance: ebonite.core.objects.core.RuntimeInstance, meta_only: bool = False)[source]¶ ” Stops instance of model service and deletes it from repository
Parameters: - instance – instance to delete
- meta_only – only remove from metadata, do not stop instance
Returns: nothing
-
delete_environment
(environment: ebonite.core.objects.core.RuntimeEnvironment, meta_only: bool = False, cascade: bool = False)[source]¶ ” Deletes environment from metadata repository and(if required) stops associated instances
Parameters: - environment – environment to delete
- meta_only – wheter to only delete metadata
- cascade – Whether should environment be deleted with all associated instances
Returns: Nothing
- meta_repo –
-
ebonite.
start_runtime
(loader=None, server=None)[source]¶ Starts Ebonite runtime for given (optional) loader and (optional) server
Parameters: - loader – loader of model to start Ebonite runtime for,
if not given class specified in
config.Runtime.LOADER
is used - server – server to use for Ebonite runtime, default is a flask-based server,
if not given class specified in
config.Runtime.SERVER
is used
Returns: nothing
- loader – loader of model to start Ebonite runtime for,
if not given class specified in
-
ebonite.
create_model
(model_object, input_data, model_name: str = None, params: Dict[str, Any] = None, description: str = None, **kwargs) → ebonite.core.objects.core.Model[source]¶ Creates Model instance from arbitrary model objects and sample of input data
Parameters: - model_object – model object (function, sklearn model, tensorflow output tensor list etc)
- input_data – sample of input data (numpy array, pandas dataframe, feed dict etc)
- model_name – name for model in database, if not provided will be autogenerated
- params – dict with arbitrary parameters. Must be json-serializable
- description – text description of this model
- kwargs – other arguments for model (see Model.create)
Returns: Model
instance
Subpackages¶
- ebonite.build package
- ebonite.client package
- ebonite.core package
- Subpackages
- ebonite.core.analyzer package
- ebonite.core.objects package
- Submodules
- ebonite.core.objects.artifacts module
- ebonite.core.objects.core module
- ebonite.core.objects.dataset_source module
- ebonite.core.objects.dataset_type module
- ebonite.core.objects.metric module
- ebonite.core.objects.requirements module
- ebonite.core.objects.typing module
- ebonite.core.objects.wrapper module
- Submodules
- Submodules
- Subpackages
- ebonite.ext package
- Subpackages
- ebonite.ext.aiohttp package
- ebonite.ext.catboost package
- ebonite.ext.docker package
- ebonite.ext.flask package
- ebonite.ext.imageio package
- ebonite.ext.lightgbm package
- ebonite.ext.numpy package
- ebonite.ext.pandas package
- ebonite.ext.s3 package
- ebonite.ext.sklearn package
- ebonite.ext.sqlalchemy package
- ebonite.ext.tensorflow package
- ebonite.ext.tensorflow_v2 package
- ebonite.ext.torch package
- ebonite.ext.xgboost package
- Submodules
- Subpackages
- ebonite.repository package
- ebonite.runtime package
- ebonite.utils package