Batch requests

The problem

Imagine we are developing a backend API for an online store, and we already have a ProductService.getInfo method for the Product entity. The frontend task is to implement the logic of adding products to the cart and refreshing product information on demand (for example, when the cart is opened, we need to update information about the products the user has added: availability, prices, descriptions, etc.).

The scenario 100% of developers follow

A backend task is created to build a separate API method that returns a collection of objects for an array of identifiers. This takes the working time of a backend developer, a QA engineer and a frontend developer, because the new method has to be covered by unit tests as well as additional regression test scenarios. All of this has to be done even though we already have a method that can return a single product by its id.

The scenario nobody follows

Send a separate API request for each product to get up-to-date information. If there are 10 products in the cart, that means 10 HTTP requests. This puts a significant load on the network and increases the user's waiting time, which makes it an unacceptable option.

The alternative using batch requests

If we can send and process batch requests, we can greatly simplify the implementation of the business logic. We can use a batch request to combine all these requests into one. We send one request to the API that contains all the product identifiers, and the server returns information about all the products in a single response. As a result, we get:

1. Universality: Using batch requests avoids creating specialized methods for every case. This simplifies the API and reduces the amount of code required.
2. Convenience for the client: The client can decide which requests to combine, which makes the API more flexible and convenient to use.
3. Less backend complexity: Instead of writing and maintaining specialized methods for different scenarios, you can use the universal batch request approach, which simplifies the backend architecture.

Example

Let's look at the request from the example above

[  
   {  
      "id":"example_1",  
      "method":"ProductService.getInfo",  
      "params":{  
         "productId": 345234  
      }  
   },  
   {  
      "id":"example_2",  
      "method":"ProductService.getInfo",  
      "params":{  
         "productId": 994234  
      }  
   }  
]

[  
   {  
      "id":"example_2",  
      "result": {  
         "id": 994234,  
         "name": "product 2",  
         "price": 11.99,  
         "balance": 28  
      }  
   },  
   {  
      "id":"example_1",  
      "result": {  
         "id": 345234,  
         "name": "product 1",  
         "price": 99.99,  
         "balance": 14  
      }  
   }  
]

Advantages of batch requests

1. Simpler backend: Instead of implementing specific logic on the backend every time, you can move it to the level of building client requests.
2. Fewer network requests: Instead of sending several HTTP requests, which can load the network and the server, only one request is used. This significantly reduces the load on the network and server resources.
3. Response time: The RPC Server returns the result of all requests, regardless of their number, in the time needed to process the longest request in the batch, because all requests are processed in parallel.
4. Lower latency: Executing several requests over one connection can reduce the overall latency, since less time is spent on establishing and closing connections.
5. Optimized client and server resource usage: Since fewer requests are processed at the same time, the load on the server decreases, allowing it to handle more requests. The client also uses fewer resources, which can be critical for mobile and constrained devices.
6. Better transaction control: When several operations need to be executed atomically (all or none), a batch request makes this easier, since all operations are executed within a single request.

Getting started

Batch requests are available out of the box. How the items are executed is defined by the batch block in the configuration — see the "Execution modes" section below.

Request order

The order of items in a batch does not matter. An error in one item does not affect the processing of the others: the response contains an error object for it and a result for the rest.

Dependent requests

Imagine that the frontend needs to get a user's email address by their identifier and then send them a message at that address. We already have the UserService.getInfo and Messenger.sendEmail methods. In the classic scenario this would be either two requests, or a separate backend method that sends an email by user id would have to be created.

Using the JsonRpcBundle library by UFO-Tech, you can create batch requests in which one request can depend on another and receives its response in order to run. In the context of the example, one request gets the user information, and the second request uses the obtained email to send the message.

[  
   {  
      "jsonrpc": "2.0",  
      "id":"example_1",  
      "method":"UserService.getInfo",  
      "params":{  
         "userId": 141  
      }  
   },  
   {  
      "jsonrpc": "2.0",  
      "id":"example_2",  
      "method":"Messenger.sendEmail",  
      "params":{  
         "email": "@FROM:example_1(email)",  
         "subject": "Welcome!",  
         "message": "Thank you for joining our service!"  
      }  
   }  
]

[  
   {  
      "id":"example_1",  
      "result": {  
         "id": 141,  
         "name": "John",  
         "email": "john.dou@example.com",  
         "status": 1  
      }  
   },  
   {  
      "id":"example_2",  
      "result": "Email for 'john.dou@example.com' sent"  
   }  
]

Dependency format: @FROM:<request id>(<result field>). Dependencies work in both execution modes.

In this example:

1. The first request gets information about the user with id 141, including their email.
2. The second request sends an email to the obtained address, using data from the first request.

Advantages of dependent requests:

1. Convenience and efficiency: Dependent requests let you perform complex operations with minimal effort while ensuring the correct order of request execution.
2. Fewer network requests: All requests are combined into one, which reduces the network load.
3. Flexibility: You can build complex request scenarios without additional configuration or backend changes.

Execution modes

# config/packages/ufo_json_rpc.yaml
ufo_json_rpc:
    batch:
        enabled: true       # false: batch requests are rejected with a single -32600 "Batch requests are disabled" error
        process: true       # true: each item in a child process; false: one by one in the current process
        process_count: 2    # process mode: how many child processes run at the same time (1–10)

The token is passed to child processes via the UFO_RPC_TOKEN environment variable and the request via stdin, so they do not show up in the process list. Child processes keep the API group and version of the route.

Item timeout

In process mode, the response of an item is awaited for a limited time: 10 seconds by default. If there is no result, an error is returned for that item. For long methods, the timeout is increased with the special $rpc.timeout parameter:

[
    {"jsonrpc": "2.0", "id": "example_1", "method": "ExampleApi.fastMethod", "params": {"someParam": "someValue"}},
    {"jsonrpc": "2.0", "id": "example_2", "method": "ExampleApi.longMethod", "params": {"someParam": "someValue", "$rpc": {"timeout": 30}}}
]

Notifications in a batch

An item without an id is a notification. According to the JSON-RPC 2.0 specification, there is no response to it:

• a notification is executed according to the batch mode, but the response contains only the other items;
• a batch of notifications only returns 204 with no body;
• a notification whose @FROM dependency failed is not executed.

Batch errors

• An empty batch [] — a single -32600 error (Can't process empty batch request).
• A batch with batch.enabled: false — a single -32600 error (Batch requests are disabled).