We got a service that has an HTTP request rate limit: less than 1 message per 10 seconds. we don’t use this service frequently,
but when we use it, we send two requests sequentially, as expected, we received few
http 429 errors.
I kind of agree that it’s my responsibility to control the rate, but I don’t want to let my code aware of these constraints, so we decided to let Google Tasks control the rate.
I don’t know much about
token bucket, by glancing the help doc, I think it will help by
--max-dispatches-per-second=0.01 (1 message / 10 seconds) with:
cloud tasks queues update my-task-queue --max-concurrent-dispatches=1 --max-dispatches-per-second=0.01
however, we noticed that HTTP 429 persists after the change, task queue log shows tasks are dispatched almost at the same time. until we checked the
Each queue has a token bucket that holds tokens, up to the maximum specified by maxBurstSize. Each time a task is dispatched, a token is removed from the bucket. Tasks will be dispatched until the queue’s bucket runs out of tokens. The bucket will be continuously refilled with new tokens based on maxDispatchesPerSecond. and this field is an
gcloud tasks queues describe my-task-queueshows the
so the bucket should have 10 tokens initially, even though I set the rate, but in my case, the first call will get run immediately because 10 tokens are available right there. read the document again, and I found:
In most cases, using the Cloud Tasks API method and letting the system set max_burst_size produces a very efficient rate for managing request bursts. In some cases, however, particularly when the desired rate is relatively slow, either using the queue.yaml method to manually set bucket_size to a small value, or setting your max_concurrent_dispatches to a small value via the Cloud Tasks API can give you more control. https://cloud.google.com/tasks/docs/configuring-queues#rate
set bucket_size to 1 using
queue.yaml, task queue log shows tasks are dispatched right at the rate I set.
That’s not all, you’d better to read this one before using queues.xml:
and these posts also help:
- a feature request: Make Max Burst Size configurable