Upload of missing file crashes script.

hugolpz commented 3 years ago

I started script to upload 10 files. One target is missing :

path: ../DragonsBot/SOP/亠-red.png

Node script (relevant section):

            try{ 
                // upload
                let result = await targetWiki.upload({
                    file_path: `../DragonsBot/SOP/${zi}-${locale}${major}.${extension}`,
                    comment: `Upload Chinese radical ${zi} in red style, raster format.`, // <------------------------ TO EDIT
                    text: wikicode,
                    ignorewarnings: 1, // overwrite or create,
                    author: '[[User:Yug]]',
                    categories:[``]
                });
            }catch(e){ console.error(e); }

The node shell/script crashes on it, returning :

get_API_parameters: Set commonswiki: path=upload
internal/fs/utils.js:259
    throw err;
    ^

Error: ENOENT: no such file or directory, open '../DragonsBot/SOP/亠-red.png'
    at Object.openSync (fs.js:461:3)
    at Object.readFileSync (fs.js:364:35)
    at get_file_object (/home/yug/Documents/DragonsBot/node_modules/cejs/application/net/Ajax.js:638:24)
    at process_next (/home/yug/Documents/DragonsBot/node_modules/cejs/application/net/Ajax.js:759:5)
    at process_next (/home/yug/Documents/DragonsBot/node_modules/cejs/application/net/Ajax.js:791:4)
    at process_next (/home/yug/Documents/DragonsBot/node_modules/cejs/application/net/Ajax.js:791:4)
    at process_next (/home/yug/Documents/DragonsBot/node_modules/cejs/application/net/Ajax.js:791:4)
    at process_next (/home/yug/Documents/DragonsBot/node_modules/cejs/application/net/Ajax.js:791:4)
    at to_form_data (/home/yug/Documents/DragonsBot/node_modules/cejs/application/net/Ajax.js:793:3)
    at get_URL_node (/home/yug/Documents/DragonsBot/node_modules/cejs/application/net/Ajax.js:1395:4) {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: '../DragonsBot/SOP/亠-red.png'
}

I assumed the script would ping me "file missing" then pass over it and move to the next target.

The script works properly when files exist.

Note/idea: if there are both successful and missing files, it may be interesting to log the list of all successfull / failed upload, to then print it when the script ends.

kanasimi commented 3 years ago

I thought if we try { .upload() } and cache the error, the program will keep running. Didn’t the library actually do this?

kanasimi commented 3 years ago

Maybe you can give some further information, so I can know what had happened?

kanasimi commented 2 years ago

Hi. Is this error still appears?

hugolpz commented 2 years ago

Yes. Still there.

Assets in ./media: letters test files available here.

Data in ./data/letters.js :

module.exports = [
    // Data relevant to media files in ../media/
    // { letter: 'a', filename: 'a-letter-test.svg', author: 'Yug' },
    // { letter: 'b', filename: 'b-letter-test.svg', author: 'Yug' }, 
    { letter: 'z', filename: 'z-letter-test.svg', author: 'Yug' }, // <---- file does not exist, will fails ! --------------------------------- !!!!!
    { letter: 'c', filename: 'c-letter-test.svg', author: 'Yug' },
    { letter: 'd', filename: 'd-letter-test.svg', author: 'Yug' },
    { letter: 'e', filename: 'e-letter-test.svg', author: 'Yug' },
    { letter: 'f', filename: 'f-letter-test.svg', author: 'Yug' },
];

Code in ./wiki-upload-many.js :

// PURPOSE: Script to upload targets using an external data file.
// Run: $node wiki-upload-many.js
const Wikiapi= require('wikiapi');
const logins = require('./logins.js'); // <----------------- credentials from there
const letters= require('./data/letters.js');  // <----------------- data from there

// Login credentials from .login*.js
var USER = logins.commons.user,
    PASS = logins.commons.pass,
    API  = logins.commons.api;

(async () => {
    // Connect
    const wiki = new Wikiapi;
    await wiki.login(USER, PASS, API);
    console.log(`Username ${USER.split('@')[0]} is connected !`);

/* *************************************************************** */
/* CORE ACTION(S) HERE : HACK ME ! ******************************* */
    // Set upload parameters, maily for licensing reasons.
    // Note: parameter `text`, filled with the right wikicode `{{description|}}`, can replace most parameters.
    let options = {
        description: 'The letter',
        author: `[[User:${USER.split('@')[0]}|]]`,
        date: new Date().toISOString().split('T')[0],
        //source_url: 'https://github.com/kanasimi/wikiapi',
        permission: 'own work',
        other_versions: '',
        other_fields: '',
        license: ['{{PD-ineligible}}'],
        categories: [`[[Category:ShufaBot test: upload]]`],
        bot: 1,
       // tags:"tag1|tag2",
    };

    for(i=0;i<letters.length;i++){
        console.log(letters[i].filename)
        // Upload file from URL
        try{
            wiki.upload({
                file_path: `./media/${letters[i].filename}`,
                filename: `Letter-${letters[i].letter}-colors2.svg`,  // default : keep filename
                comment: `Upload colorful letter ${letters[i].letter} for latin alphabet.`,
                ignorewarnings: 1,  // overwrite
                ...options,
                description: `The letter ${letters[i].letter}, in black on white.`,
                author: `[[User:${USER.split('@')[0]}|]]`,
            });
        }catch(e){ console.error('Fails upload: '+ e); }
    }
/* END CORE ****************************************************** */
/* *************************************************************** */

})();

Error message:

node wiki-upload-many.js 
get_API_parameters: Set commonswiki: path=query+siteinfo
Username ShufaBot is connected !
z-letter-test.svg
c-letter-test.svg
d-letter-test.svg
e-letter-test.svg
f-letter-test.svg
get_API_parameters: Set commonswiki: path=upload
node:internal/fs/utils:344
    throw err;
    ^

Error: ENOENT: no such file or directory, open './media/z-letter-test.svg'
    at Object.openSync (node:fs:585:3)
    at Object.readFileSync (node:fs:453:35)
    at get_file_object (/home/yug/Documents/WikiapiJS-Eggs/node_modules/cejs/application/net/Ajax.js:638:24)
    at process_next (/home/yug/Documents/WikiapiJS-Eggs/node_modules/cejs/application/net/Ajax.js:759:5)
    at process_next (/home/yug/Documents/WikiapiJS-Eggs/node_modules/cejs/application/net/Ajax.js:791:4)
    at process_next (/home/yug/Documents/WikiapiJS-Eggs/node_modules/cejs/application/net/Ajax.js:791:4)
    at process_next (/home/yug/Documents/WikiapiJS-Eggs/node_modules/cejs/application/net/Ajax.js:791:4)
    at process_next (/home/yug/Documents/WikiapiJS-Eggs/node_modules/cejs/application/net/Ajax.js:791:4)
    at process_next (/home/yug/Documents/WikiapiJS-Eggs/node_modules/cejs/application/net/Ajax.js:791:4)
    at to_form_data (/home/yug/Documents/WikiapiJS-Eggs/node_modules/cejs/application/net/Ajax.js:793:3) {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: './media/z-letter-test.svg'
}

Node.js v17.0.1

On Commons :

All the files up to the missing one are uploaded successfully. Script stops at missing file. There is no difference if using await wiki.upload() or wiki.upload().
If the missing file is first, then NO file is uploaded, even if letters c, d, e, f's files existing locally and are available for upload.
If there is no missing file z in the data, all target and existing files are uploaded correctly to Commons.

Expected behavior (?): Since we are coding bots and aiming at automation, I expected such function to log an error for missing local/url files but to continue its lifecycle uploading all available files.

hugolpz commented 2 years ago

@kanasimi, what is the new expected behavior ?

kanasimi commented 2 years ago

It should continue uploading other files now.

kanasimi / wikiapi

Upload of missing file crashes script. #43