![]() ![]() This article isn't to try and put you off running ad hoc NLP pipelines in lambda, on the contrary, it hopes to help make you aware of these constraints and some ways to overcome them (or alternatively, help you to live on the edge by choosing to ignore from a place of understanding). Get and load the package containing the lambda code from external persistent storage (e.g.When a container starts from a cold state, the function needs to: What is a cold start?Ī cold start occurs when the container is shut down and then has to be restarted when the lambda function is invoked, typically this happens after ~5 mins of inactivity.Ī lot has already been written about cold starts, so this article won’t provide a detailed guide (I recommend you check out this article for that). Run the function’s handler method/function. step 4 always happen whenever you invoke a lambda function (it’s your code!), but steps 1–3 only occur for cold starts. ![]() ![]() As the setup stages take place entirely in AWS, they are outside of our control. We can optimise to our heart’s content in step 4, but steps 1–3 can still arbitrarily hit us with ~10 second+ latencies. This is obviously a problem for synchronous APIs. Took an arbitrary text input from an HTTP request.Downloaded an NLP model from S3 (~220mb).Performed NLP on the input using the model.Returned the serialised result to the caller.Downloading the model from s3 on each invocation could take between 15–20 seconds. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |