My suspicion is that the our internal package requires os as a dependency to open a file, and that causes some kind of clash with AWS.

Unlike in AWS Lambda code, in Sagemaker training jobs using custom Python code in Docker containers logging won’t work unless a stream handler is specifically added to the logger.

At the end of a Dockerfile that defines a container for a Sagemaker training job, you have to define the SAGEMAKERPROGRAM environment variable. It needs to point directly to…

To pass arguments to training code in Sagemaker by itself or via a Step Function, set up an argument parser in the Python script with keyword arguments.

With SageMaker training jobs, you specify an “input channel” and configure it in the job definition. Each channel is a dictionary that specifies the data split. Each of these…