Request — Support bulk entries creation/updates

[Aurelsicoko]

This issue has been opened because we (as the core team) are looking for contributors to help us co-develop the feature.

Link to the roadmap

Motivations

A generated API exposes a create route (POST). The route only supports creating one entry at a time. We suggest that we could enhance the current create route or to create a new route to support bulk entries creation.

Tasks

1. Find a contributor
2. Define the needs
3. Discuss a solution and technical implementation
4. Ask for design
5. Submit RFC

Risks

• Don't forget to support bulk updates.
• Performance issues

[basavarajdodamani]

Any updates, if this feature has been implemented on alpha version - 3.1.1 Tried with provided controller logic, which did not work for me. Could you please share Services snippet as well to make it work. P.S: I have just started learning Strapi.

[derrickmehaffy]

@basavarajdodamani nope no work has started, this repo is just for RFCs or "Request for comments". Nothing will be implemented until the RFC is approved and in certain cases, such as this one, someone volunteers to work on it or the Strapi team have time to implement it.

[darron1217]

+ we also need bulk deletes

[m-emre-yalcin]

@darron1217 Strapi already has that functionality in the admin panel. You can check "modules/strapi-plugin-content-manager/services/ContentManager.js" path. This function below can help you to improve your own controller i guess.

deleteMany(model, ids, query) {
    const { primaryKey } = strapi.query(model);

    return strapi.entityService.delete(
      {
        params: {
          _limit: 100,
          ...query,
          _where: _.concat({ [`${primaryKey}_in`]: ids }, query._where || {}),
        },
      },
      { model }
    );
  },

example request: Request URL: http://localhost:1337/content-manager/explorer/deleteAll/plugins::users-permissions.user?0=11&1=14 // delete 11 and 14

[darron1217]

@m-emre-yalcin Thanks, but I meant Auto Generated REST apis. Sure we can make it manually, but if there's plan to add bulk edit & update for default, it would be nice to have bulk delete endpoint too.

[chloesun]

Is there a workaround to do bulk create/edit before this feature is implemented? I'd like to see some examples, thanks all!

[Aurelsicoko]

Is there a workaround to do bulk create/edit before this feature is implemented? I'd like to see some examples, thanks all!

A possible (not the best) implementation would be to create a new endpoint in the routes.json file of a generated API.

 {
      "method": "DELETE",
      "path": "/articles/bulk/:ids",
      "handler": "article.bulkDelete",
      "config": {
        "policies": []
      }
 }

Then in your controller, you create a new action that maps the delete service to remove an entry from the database (e.g Article collection type).

module.exports = {
  async bulkDelete(ctx) {
    const ids = ctx.params.ids.split(',');

    return Promise.all(ids.map(id => strapi.services.article.delete({ id })));
  }
};

Finally, you need to make a DELETE HTTP request like this DELETE http://localhost:1337/articles/bulk/1,2,3,4,5.

PS: Make sure you've opened the permissions in the Users & Permissions > Roles > Public page, if you aren't authenticated.

[chloesun]

@Aurelsicoko Thanks for the code sample, is it possible to show some examples of POST and PUT bulk request? Thanks a lot!!!

[Aurelsicoko]

@chloesun You can follow the same logic whatever the endpoint is

 {
      "method": "PUT",
      "path": "/articles/bulk/:ids",
      "handler": "article.bulkUpdate",
      "config": {
        "policies": []
      }
 }

const { parseMultipartData, sanitizeEntity } = require('strapi-utils');

module.exports = {
  async bulkUpdate(ctx) {
    const ids = ctx.params.ids.split(',');
    let entities;

    if (ctx.is('multipart')) {
      // Hard to support for bulk action
      const { data, files } = parseMultipartData(ctx);

      entities = await Promise.all(
        ids.map(id => strapi.services.article.update({ id }, data, { files }))
      );
    } else {
      entities = await Promise.all(
        ids.map(id => {
          strapi.services.article.update({ id }, ctx.request.body)
        })
      );
    }

    return entities.map(entity => sanitizeEntity(entity, { model: 'article' }));
  }
};

Warning: It works but I won't recommend using it in production, the error handling has to be improved first, I didn't check if it works when we upload files, etc. I'm just sharing the main concept and pattern here.

[derrickmehaffy]

@Aurelsicoko do you think it would be better to handle the array of IDs as a query parameter and not a dynamic variable?

Something like PUT /articles/bulk?ids=[1,2,3,4,5] or something. Trying to think of how best to handle that in the scope of something like OpenAPI or OpenCRUD (REST / GraphQL)

[Aurelsicoko]

Yep, the ID could be also pass through the body of the PUT HTTP query. I don't know if there is any limit of length on a URL? I was thinking about a potential issue for Mongo IDs because they are pretty long.

I don't know if the OpenAPI has a recommendation for bulk queries?

[derrickmehaffy]

Yep, the ID could be also pass through the body of the PUT HTTP query. I don't know if there is any limit of length on a URL? I was thinking about a potential issue for Mongo IDs because they are pretty long.

I don't know if the OpenAPI has a recommendation for bulk queries?

2000 characters generally (depends on the browser)

And that would work for creation and updates (sending in the body) but I don't believe the DELETE method allows for body :thinking:

[baermathias]

@derrickmehaffy it would also be great if not only the API would support bulk operations, but also the admin dashboard ui. There are many cases, where you as a business user would want to select 2000 voucher codes or whatever collections at the same time and then change e.g. the expiration date on all 2000 voucher codes at the same time. You don't want to open 2000 entries one by one.

[derrickmehaffy]

@derrickmehaffy it would also be great if not only the API would support bulk operations, but also the admin dashboard ui. There are many cases, where you as a business user would want to select 2000 voucher codes or whatever collections at the same time and then change e.g. the expiration date on all 2000 voucher codes at the same time. You don't want to open 2000 entries one by one.

Afaik at this time only the query layer will have functions to do bulk operations. You could use these query functions to build plugins or write custom actions but we won't add them ourselves as in many cases we don't recommend handing the bulk operations on the server (load balancing ect).

But I understand there could be additional bulk operations added to the admin, although it's highly unlikely we would add a bulk edit in the short term and more likely we could add something more simple like bulk publish.

[itsmatteomanf]

@derrickmehaffy we could add something more simple like bulk publish

That would be a big step, as now it's causing multiple webhooks and it's not what one could call atomic, it wouldn't still be global publish in all possible types, which could be an issue given relations, but it would work way better than it now does.

[aleandriproton]

When are we going to have "Publish" as part of the bulk actions? Is there a plugin for that? A work around?

[TheBit]

When are we going to have "Publish" as part of the bulk actions? Is there a plugin for that? A work around?

As well as bulk "Unpublish". Coz draft-n-publish flow allows us to enable our clients to "remove" features from the site, without deleting them from CMS. But without bulk - this looks like an incomplete feature.

[TheBit]

@aleandriproton bulk publish was recently implemented https://github.com/strapi/strapi/releases/tag/v4.11.0