Multiple file uploads is not working

[SiriSuri]

"node": "^12.16" "sails-mongo": "^1.2.0" "sails": "^1.2.3", "sails-hook-grunt": "^4.0.0", "sails-hook-orm": "^2.1.1", "sails-hook-sockets": "^2.0.0",

Above are the versions i am using. my requirements is I will be receiveing array of files from HTML along with 5 arrays of text fields.I am uploading files to my images folder and saving the file name to db using the following code.

incdesc=req.body.incdesc; idate=req.body.idate; itime=req.body.itime; sysaffected=req.body.sysaffected; solution=req.body.solution; //prevmes:preventive measures prevmes=req.body.prevmes; var a=''; console.log(incdesc,idate,itime,sysaffected,solution,prevmes) var f= req.file('ifile') console.log(req.session.uid); f.upload({dirname:'../../assets/images/logs'}, function whenDone(err, files) { if (err) return res.serverError(err); else if(files.length==0){ message="No file uploaded-log"; console.log(message); } else{ for(i=0;i<files.length;i++){ a=files[i].fd.toString();var start=a.lastIndexOf('\'); a=a.substring(start+1) console.log(a) Incident.create({username:username,recorddate:date,type:type,month:month,year:year,inc:inc,incdesc:incdesc[i+1],idate:idate[i+1],itime:itime[i+1],sysaffected:sysaffected[i+1],solution:solution[i+1],prevmes:prevmes[i+1],ifile:a}).exec(function (err,fij){ if (err) return res.serverError(err); console.log(fij)
}); }} });

This is working fine when i upload files of small size. but as the file size is increasing to 1 mb, it is able to save the file location, but the other text fields are missing. That is like only the first record is getting created with all params. from second record, only the file location is being saved and remaining fields as empty. I dont understand where i am doing wrong

[sailsbot]

@SiriSuri Thanks for posting! We'll take a look as soon as possible.

In the mean time, there are a few ways you can help speed things along:

• look for a workaround. (Even if it's just temporary, sharing your solution can save someone else a lot of time and effort.)
• tell us why this issue is important to you and your team. What are you trying to accomplish? (Submissions with a little bit of human context tend to be easier to understand and faster to resolve.)
• make sure you've provided clear instructions on how to reproduce the bug from a clean install.
• double-check that you've provided all of the requested version and dependency information. (Some of this info might seem irrelevant at first, like which database adapter you're using, but we ask that you include it anyway. Oftentimes an issue is caused by a confluence of unexpected factors, and it can save everybody a ton of time to know all the details up front.)
• read the code of conduct.
• if appropriate, ask your business to sponsor your issue. (Open source is our passion, and our core maintainers volunteer many of their nights and weekends working on Sails. But you only get so many nights and weekends in life, and stuff gets done a lot faster when you can work on it during normal daylight hours.)
• let us know if you are using a 3rd party plugin; whether that's a database adapter, a non-standard view engine, or any other dependency maintained by someone other than our core team. (Besides the name of the 3rd party package, it helps to include the exact version you're using. If you're unsure, check out this list of all the core packages we maintain.)

  ---

Please remember: never post in a public forum if you believe you've found a genuine security vulnerability. Instead, disclose it responsibly.

For help with questions about Sails, click here.

[crh3675]

Not sure how relevant with version 1.0 but we had similar issue and needed to adjust the config/http.js by overriding the defaults.

      multipartParser: require('multer')({
         dest: './uploads',
         limits: {
            fieldSize: 1000 * 1000 * 5,
            fileSize: 1000 * 1000 * 50
         }
      }).any(),

      urlencodedParser: bodyParser.urlencoded({
         limit: '50mb',
         extended: true
      }),

      jsonParser: bodyParser.json({limit: '50mb'}),

      rawParser: bodyParser.raw({limit: '50mb'}),

      textParser: bodyParser.text({limit: '50mb'})

And top of file:

module.exports.http = {
   /****************************************************************************
    *                                                                           *
    * Express middleware to use for every Sails request. To add custom          *
    * middleware to the mix, add a function to the middleware config object and *
    * add its key to the "order" array. The $custom key is reserved for         *
    * backwards-compatibility with Sails v0.9.x apps that use the               *
    * `customMiddleware` config option.                                         *
    *                                                                           *
    ****************************************************************************/

   middleware: {

      /***************************************************************************
       *                                                                          *
       * The order in which middleware should be run for HTTP request. (the Sails *
       * router is invoked by the "router" middleware below.)                     *
       *                                                                          *
       ***************************************************************************/

      order: [
         'startRequestTimer',
         'cookieParser',
         'session',
         'myRequestLogger',
         'urlencodedParser',
         'jsonParser',
         'rawParser',
         'textParser',
         'multipartParser',

[SiriSuri]

Hi this is what i made in my http.js based on your suggestion. /**

• HTTP Server Settings
• (sails.config.http)
• Configuration for the underlying HTTP server in Sails.
• (for additional recommended settings, see config/env/production.js)
• For more information on configuration, check out:
• https://sailsjs.com/config/http */

module.exports.http = {

/****

• *
• Sails/Express middleware to run for every HTTP request. *
• (Only applies to HTTP requests -- not virtual WebSocket requests.) *
• *
• https://sailsjs.com/documentation/concepts/middleware *

• * ****/

  middleware: {

  /***

  • *
  • The order in which middleware should be run for HTTP requests. *
  • (This Sails app's routes are handled by the "router" middleware below.) *
  • * ***/ order: [ 'startRequestTimer', 'cookieParser', 'session', 'myRequestLogger', 'urlencodedParser', 'jsonParser', 'rawParser', 'textParser', 'multipartParser', ],

  // order: [ // 'cookieParser', // 'session', // 'bodyParser', // 'compress', // 'poweredBy', // 'router', // 'www', // 'favicon', // ],

  /***

  • *
  • The body parser that will handle incoming multipart HTTP requests. *
  • *
  • https://sailsjs.com/config/http#?customizing-the-body-parser *
  • * ***/

  multipartParser: require('multer')({ dest: './assets/images/logs', limits: { fieldSize: 1000 1000 5, fileSize: 1000 1000 50 } }).any(),

  urlencodedParser: multipartParser.urlencoded({ limit: '50mb', extended: true }),

  jsonParser: bodyParser.json({limit: '50mb'}),

  rawParser: bodyParser.raw({limit: '50mb'}),

  textParser: bodyParser.text({limit: '50mb'})

  // bodyParser: (function _configureBodyParser(){ // var skipper = require('skipper'); // var middlewareFn = skipper({ strict: true}); // return middlewareFn; // })(),

  },

};

But it is showing errors. error is : ReferenceError: bodyParser is not defined can you help me to resolve this?

[crh3675]

You need to install multer (for multipart) in order to override. I provided a full config that worked for me.

npm install multer --save

Here is a full file for you:

/**
 * HTTP Server Settings
 * (sails.config.http)
 *
 * Configuration for the underlying HTTP server in Sails.
 * (for additional recommended settings, see `config/env/production.js`)
 *
 * For more information on configuration, check out:
 * https://sailsjs.com/config/http
 */
const multer = require('multer');
const bodyParser = require('body-parser');

// Set max upload sizes
const uploadsPath = './uploads';
const maxUploadSize = 50; // mb
const maxUploadFields = 1000;
const maxUploadOptions = {
  limit: `${maxUploadSize}mb`
};

// Explicit parsers to control size of
// uploaded data
const parsers = {
  // Review documentation on multer for more advanced configuration
  // http://expressjs.com/en/resources/middleware/multer.html
  multipart: multer({
    dest: uploadsPath,
    limits: {
      fieldSize: maxUploadFields,
      fileSize: 1000 * 1000 * maxUploadSize
    }
  }).any(),

  urlencoded: bodyParser.urlencoded({
    limit: maxUploadOptions.limit,
    extended: true
  }),

  json: bodyParser.json(maxUploadOptions),
  text: bodyParser.text(maxUploadOptions),
  raw: bodyParser.raw(maxUploadOptions)
};

module.exports.http = {

  /****************************************************************************
  *                                                                           *
  * Sails/Express middleware to run for every HTTP request.                   *
  * (Only applies to HTTP requests -- not virtual WebSocket requests.)        *
  *                                                                           *
  * https://sailsjs.com/documentation/concepts/middleware                     *
  *                                                                           *
  ****************************************************************************/

  middleware: {

    /***************************************************************************
    *                                                                          *
    * The order in which middleware should be run for HTTP requests.           *
    * (This Sails app's routes are handled by the "router" middleware below.)  *
    *                                                                          *
    ***************************************************************************/

    // order: [
    //   'cookieParser',
    //   'session',
    //   'bodyParser',
    //   'compress',
    //   'poweredBy',
    //   'router',
    //   'www',
    //   'favicon',
    // ],

    order: [
      'startRequestTimer',
      'cookieParser',
      'session',
      // Explicit body parser updates
      'urlencodedParser',
      'jsonParser',
      'rawParser',
      'textParser',
      'multipartParser',
      // body parser done
      'compress',
      'router',
      'www',
      'favicon'
    ],

    urlencodedParser: parsers.urlencoded,
    multipartParser: parsers.multipart,
    jsonParser: parsers.json,
    textParser: parsers.text,
    rawParser: parsers.raw
  },
};

[SiriSuri]

Can you please suggest any tutorial through which i can learn file uploading thorigh multer. I tried ur code. No errors in http.js. but there is an error in my code. my code is: vapt1file=req.file("vapt1file"); vapt1file.upload({dirname:'../../assets/images/logs'}, function onUpload(err, files) { if (err) { message="error occured.please try again"; console.log(err); FlashService.error(req,message); res.redirect('/dashboard/home') // res.view('',{rid:req.params.id}) }
else if(files.length==0){ // var a= await Vapt.find({vno:id}); // console.log(a[0].loglocation); message="No file uploaded-log"; console.log(message); logid= Vapt.update({id:id}).set({vapt1file:'',vapt1:false,type:type,recorddate:date}).exec(function(err){ if(err) console.log(err) else console.log('success'); }) } else{ loglocation=files[0].fd.toString();var start=loglocation.lastIndexOf('\'); loglocation=loglocation.substring(start+1) console.log(loglocation) Vapt.update({id:id}).set({vapt1file:loglocation,vapt1:true}).exec(function(err,user){ if(err) console.log(err) else console.log(user) }) }});

this is the error. req.file is not a function.

May be I should modify my code as per multer? Can you suggest that Please?

[crh3675]

Try req.files instead.

.any() Accepts all files that comes over the wire. An array of files will be stored in req.files.

That would be an array of objects that you can iterate through:

{
  "fieldName": 'vapt1file',
  "originalName": "orig name"
  "path": 'the uploaded path';
  "size": 'the size',
  "mimeType": 'some type' 
}

You can access using a filter:

const vapt1file = req.files.find(f => f.fieldname === 'vapt1file');

Not sure of your entire files contents but I cleaned up in order to help. One thing, when posting code examples, please apply code-formatting as it is much easier to read.

const vapt1file = req.files.find(f => f.fieldname === 'vapt1file');
const updates = {
  vapt1file: '',
  vapt1: false,
  type: type, // IS THIS DEFINED SOMEWHERE ELSE?
  recorddate: date // IS THIS DEFINED SOMEWHERE ELSE?
};

if (vapt1file) {
  updates.vapt1file = vapt1file.path;
  uppdates.vapt1 = true;
} else {
  sails.log.info('No file uploaded')
}

Vapt.update({id:id}).set(updates).exec(function(err){
  if(err) {
    sails.log.error(err);
    FlashService.error(req, err.toString());
    res.redirect('/dashboard/home');
  } else {
    sails.log.info('success');
    res.redirect('/dashboard/home');
  }
});

[SiriSuri]

Hi.. Thanks for the suggestion. I am facing errors with req.files also. the error is: req.files is not a function I just didnt understand how to use multer in sails js. This is what I tried. Being new to JS, I cant understand anything clearly.

const upload=require("multer");
incdesc=req.body.incdesc;
idate=req.body.idate;
itime=req.body.itime;
sysaffected=req.body.sysaffected;
solution=req.body.solution;
prevmes=req.body.prevmes;
 var f= req.files('ifile')
            f.upload(function whenDone(err, files) {
                      if (err)  return res.serverError(err);
                      else if(files.length==0){
                         message="No file uploaded-log";
                         console.log(message);
                       }
                       else{
                       for(i=0;i<files.length;i++){
                       a=files[i].fd.toString();var start=a.lastIndexOf('\\');
                       a=a.substring(start+1)

                Incident.create({incdesc:incdesc[i+1],idate:idate[i+1],itime:itime[i+1],sysaffected:sysaffected[i+1],solution:solution[i+1],prevmes:prevmes[i+1],ifile:a}).exec(function (err,fij){
                        if (err) return res.serverError(err); 
                        console.log(fij)   
             });   }}
            });

in your code, you didnot use any upload function. How will it work?

[crh3675]

Does your HTML

have the correct enctype?

<form method="post" enctype="multipart/form-data">

[crh3675]

Are you running this process in a controller file? Can you post the entire file?

[SiriSuri]

yeah I am running it in a control file. I am having multipart/form-data.

const upload=require("multer");
module.exports = {
saveincident: function(req,res){
if(req.session.authenticated){
  var incdesc=[];var idate=[];var itime=[];var sysaffected=[];var solution=[]; var prevmes=[];
  var message;
var year=new Date().getFullYear();
            var username=req.session.username;
            var month=new Date().getMonth()-1;
            if(month<0) month=11;
            var date = new Date().toISOString().slice(0,10);
            var type=req.body.type;
            var inc=req.body.inc;
            if(inc=='true') inc=true;
            if(inc=='false') inc=false;
            if(inc){
              console.log("in if")
              incdesc=req.body.incdesc;
              idate=req.body.idate;
                     itime=req.body.itime;
                     sysaffected=req.body.sysaffected;
                     solution=req.body.solution;
                    //prevmes:preventive measures
                     prevmes=req.body.prevmes;
                          var a='';
                     console.log(incdesc,idate,itime,sysaffected,solution,prevmes)
                    var f= req.files('ifile')
            f.upload(function whenDone(err, files) {
                      if (err)  return res.serverError(err);
                      else if(files.length==0){
                         message="No file uploaded-log";
                         console.log(message);
                       }
                       else{
                       for(i=0;i<files.length;i++){
                       a=files[i].fd.toString();var start=a.lastIndexOf('\\');
                       a=a.substring(start+1)
                       console.log(a)
                       Incident.create({username:username,recorddate:date,type:type,month:month,year:year,inc:inc,incdesc:incdesc[i+1],idate:idate[i+1],itime:itime[i+1],sysaffected:sysaffected[i+1],solution:solution[i+1],prevmes:prevmes[i+1],ifile:a}).exec(function (err,fij){
                        if (err) return res.serverError(err); 
                        console.log(fij)   
             });   }}
            });
          }
         else{
          console.log("in else")
           Incident.create({inc:inc,username:username,recorddate:date,type:type,month:month,year:year}).exec(function (err,fij){
            if (err) return res.serverError(err); 
            console.log(fij)   
 }); 
         }
         res.redirect('/dashboard/home');
        }
          else{
            message="Please Login to View the page";
            FlashService.error(req,message);
           res.redirect('/')  }
            },
},

This is my controller function. How to incorporate multer in this?

[crh3675]

req.files is not a function, it is an Array. You need to get the file object from out of the array. Meaning the lines where you get the file info should be:

var f = req.files.find(f => f.fieldname === 'ifile');

Which will return the object. You also do not need this line const upload=require("multer"); and you don't need to use the f.upload code as the file will already be in an uploaded directory.

[SiriSuri]

ok so in controller I dont need to do anything about uploading.. Do i need to do anything in my html i.e. ejs in my case?

[crh3675]

Upload errors will automatically be sent back to the ExpressJS routing framework so you do not need to handle upload errors. As long as you have correct enctype and file input with name <input type="file" name="ifile"/> then you should have access to that uploaded file in req.files Array.

[SiriSuri]

Files are getting stored in my directory automatically. But there is no extension for the files. I was unable to open those files. I wan t to display the pdf files in my browser after saving it in my server. But no extesion is being saved. How can I do that

[crh3675]

I believe that is a security feature. You will need to rename manually before saving to your database. So replace that previous upload functionality with a rename instead.

const path = require('path');
const fs = require('fs');
const ext = path.extname(f.originalname);
const savePath = f.path + ext;
fs.rename(f.path, savePath, (err) {
   if (err) sails.log.error(err);
});

File names are randomized to prevent overwrite on the server.

[SiriSuri]

Thank you so much.. Its working fine..

[crh3675]

Glad to have helped!