Error sending local image file as photo message

outlying commented 6 months ago

NodeRed version: 3.1.0

node-red-contrib-telegrambot version: 15.1.7

According to documentation, providing a local file path to payload.content with payload.type=photo should handle the file and attach it as a photo to the message

But it is not working, I recreated this example as simply as I could and it is still failing

[
    {"id":"7c1da93d2435f4c3","type":"telegram sender","z":"12730e0f9d9df5f8","name":"Send to maintenance channel","bot":"58b199da283591e4","haserroroutput":false,"outputs":1,"x":610,"y":660,"wires":[[]]},
    {"id":"df07aa470eb7dce4","type":"function","z":"12730e0f9d9df5f8","name":"function 4","func":"const messagePayload = {\n    chatId: -1001122334455,\n    type: \"photo\",\n    content: \"/config/www/snapshots/printer_snapshot.jpg\",\n};\n\nmsg.payload = messagePayload;\n\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":360,"y":660,"wires":[["7c1da93d2435f4c3"]]},
    {"id":"24bc71bf519a2260","type":"inject","z":"12730e0f9d9df5f8","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":180,"y":660,"wires":[["df07aa470eb7dce4"]]},
    {"id":"58b199da283591e4","type":"telegram bot","botname":"RandomBotName123","usernames":"","chatids":"","baseapiurl":"","updatemode":"polling","pollinterval":"300","usesocks":false,"sockshost":"","socksport":"6667","socksusername":"anonymous","sockspassword":"","bothost":"","botpath":"","localbotport":"8443","publicbotport":"8443","privatekey":"","certificate":"","useselfsignedcertificate":false,"sslterminated":false,"verboselogging":false}
]

The error I get is:

Caught exception in sender node:
Error: ETELEGRAM: 400 Bad Request: invalid file HTTP URL specified: URL host is empty
when processing message: 
{"_msgid":"6bc61ad22941423b","payload":{"chatId":-1001122334455,"type":"photo","content":"/config/www/snapshots/printer_snapshot.jpg","options":{"chat_id":-1001122334455,"photo":"/config/www/snapshots/printer_snapshot.jpg"}},"topic":""}

Sometimes, like 6 months ago, I used a similar setup to send videos and it worked back then, I didn't use it for long but it basically did almost the same thing, record short video, save it locally, set content to the local file path and send video type message - and it worked, but that was probably version 12 something of the telegram node.

Sending normal text messages works without any issues.

windkh commented 6 months ago

There should be an example in the examples folder. Did you try that? If it still fails then something could be wrong with the path

outlying commented 6 months ago

OK, I just did, while the example is working as soon as I replace the example image URL with the local path it has the same issue as I reported.

Even more, even if I use an external URL for the same file it still fails, but I can access this file via browser, it is public and I checked if I can access it via cellular connection to roll out the possibility of some weird local network behavior.

If I use the local path I get this error:

Caught exception in sender node:
Error: ETELEGRAM: 400 Bad Request: invalid file HTTP URL specified: URL host is empty

If I use an external URL I get this:

Caught exception in sender node:
Error: ETELEGRAM: 400 Bad Request: wrong file identifier/HTTP URL specified

What bothers me is the fact that video upload is not working anymore, I found the flow I used half a year ago and it suffers the same issue. I struggled with this file setup back then https://github.com/windkh/node-red-contrib-telegrambot/issues/316 but I simply solved it by providing a local path which worked, I'm still running the same instance.

windkh commented 6 months ago

I guess it would be best if we debug that together in a meeting

outlying commented 6 months ago

I guess it would be best if we debug that together in a meeting

OK, but Christmas time is coming, I hoped that maybe that was some obvious mistake in my configuration, or maybe some more common problem.

I will try to work a bit on this issue, if I solve it I will post the solution and what was the source of the problem, but if I don't solve we can arrange a meeting after New Year. This is not urgent for me, for sure can wait another 2 and something weeks.

I will keep this issue open for now.

windkh commented 5 months ago

@outlying any news?

outlying commented 5 months ago

@windkh unluckily no, I checked a few things,

One thing I did is I provided URL to some random image I found on the Internet, and it worked - at least I know that the way I'm using those nodes is correct.

But the problem remained the same: if I replace this working URL I found with my local file path (or my external URL for the same file) I get error.

I also tried something like:

curl -v -F "chat_id=XXX" -F photo=@/tmp/img.jpg https://api.telegram.org/botXXX/sendPhoto

from HA cli and this worked, if I use telegram bot node for the same local file path (/tmp/img.jpg) and chat ID it does not work and gives me error.

... and just updated my HA, node-red pallet modules - but the issue remains

windkh commented 5 months ago

you are running under linux?

outlying commented 5 months ago

I'm running it on Raspberry Pi 4, 4GB RAM version, Home Assistant OS put directly on SD card - pretty standard setup

Bobo-amg commented 1 month ago

Any movement on this? I have the exact same problem. This used to work. Something has changed somewhere, in the last 12 months I think.

I am not running HA, just Raspbian with NR (latest)

The error I see is: Caught exception in sender node: Error: ETELEGRAM: 400 Bad Request: IMAGE_PROCESS_FAILED when processing message: {"_msgid":"523d8febc0d7dda2","topic":"test","payload":{"chatId":xxxxxxxxx"type":"photo","content":"/home/pi/camera.jpg","options":{"chat_id":xxxxxxxxx,"photo":null}}}

This is using the following in a Function Node in NR: msg.payload = { chatId: xxxxxxxxx, type: "photo", content: "/home/pi/camera.jpg" }; return msg;

This is a serious issue. Please help.

Thanks

outlying commented 1 month ago

@Bobo-amg hey, not in my case, unluckily this is not high on my priority list, little personal time and I'm not JS proficient so for me personally it is difficult to investigate this one.

However, bare in mind that this Node Red project is basically just a wrapper for node-telegram-bot-api, it is still possible that this issue is a dependency issue - but I cannot confirm that

windkh commented 1 month ago

@outlying @Bobo-amg If you want to we can debug this together in a teams meeting. I can not reproduce it here on my machine.

outlying commented 1 month ago

@windkh I can, I'm usually available between 9:00 and 15:00 on weekdays however it depends on specific time, and most of the days starting 23:00 CEST

Actually I can join teams now but no pressure

Bobo-amg commented 1 month ago

@Bobo-amg If you want to we can debug this together in a teams meeting. I can not reproduce it here on my machine.

Would love too however I'm sure I'm on the other side of the world to you so will need some time to organise.

The fact you cannot reproduce gives me some hope. Can you advise what NR, nodejs and telegrambot versions you are running?

Thanks

Bobo-amg commented 1 month ago

I've just noticed that the images that seem to be causing the issue are just coming from a raspberry pi camera. So, I guess that is where the problem lies. How they are being encoded perhaps?

windkh commented 1 month ago

@Bobo-amg If you want to we can debug this together in a teams meeting. I can not reproduce it here on my machine.

Would love too however I'm sure I'm on the other side of the world to you so will need some time to organise.

The fact you cannot reproduce gives me some hope. Can you advise what NR, nodejs and telegrambot versions you are running?

Thanks

where do you live?

Bobo-amg commented 1 month ago

Australia. You?

windkh commented 1 month ago

germany

Bobo-amg commented 1 month ago

I spent a few weeks in Germany. Loved the Hofbräuhaus in Munich.

outlying commented 1 month ago

I've just noticed that the images that seem to be causing the issue are just coming from a raspberry pi camera. So, I guess that is where the problem lies. How they are being encoded perhaps?

I don't think so, I actually used this integration with some external camera without issues, I stopped using it for few weeks and when I resumed it, it simply didn't work anymore.

I tested your encoding hypothesis, generally my example look like this

Where each function is just like this

I tested 5 variants here where I provided file path or URL as a source for image, for URL I used my own HA as a host and I also used some random cat image, all of URLs are accessible from my HA/NoneRed instance, file path is also correct

I will start with what works, this works (no issues):

https://t4.ftcdn.net/jpg/00/97/58/97/360_F_97589769_t45CqXyzjz0KXwoBZT9PRaWGHRk5hQqQ.jpg

But anything else is not, I used the same cat image and put it on my HA to test if the encoding is the issue, so none of below worked

https://192.168.1.10:8123/local/cat_img.jpg
/config/www/cat_img.jpg
https://192.168.1.10:8123/local/snapshot_camera.jpg
/config/www/snapshot_camera.jpg

Well all of those end with the same error message:

Caught exception in sender node:
Error: ETELEGRAM: 400 Bad Request: wrong file identifier/HTTP URL specified
when processing message: 
{"_msgid":"504f1dda4ed1b59e","payload":{"chatId":xxxx,"type":"photo","content":"https://192.168.1.10:8123/local/snapshot_camera.jpg","options":{"chat_id":xxxx,"photo":"https://192.168.1.10:8123/local/snapshot_camera.jpg"}},"topic":""}

...

OK, I checked one more thing, because now it appeared to me that all of those file are not being accessible from outside my network, I used an URL that points from the outside to one of my images and it worked

But this is strange because I swear it was working with local files without problems in the past, having my camera images accessible from the outside of the network is not ideal solution although I can implement some sort of "send and delete" flow

Bobo-amg commented 1 month ago

I had noticed some comments that also reference your idea that "local" files are causing an issue. However I have another data point to add.

Doing the following FAILS with above error message: take image from RPi Zero 2 / camera transfer image from Zero 2 to local NR server via Python Requests POST and save image send image to Telegram via node-red-contrib-telegrambot

Doing the following SUCCEEDS: take image from RPi Zero 2 / camera send image from Zero 2 to local NR server via SFTP send image to Telegram via node-red-contrib-telegrambot

So perhaps these are different issues with similar symptoms, as I notice that my error message is slightly different to yours.

windkh commented 1 month ago

I've just noticed that the images that seem to be causing the issue are just coming from a raspberry pi camera. So, I guess that is where the problem lies. How they are being encoded perhaps?

I don't think so, I actually used this integration with some external camera without issues, I stopped using it for few weeks and when I resumed it, it simply didn't work anymore.

I tested your encoding hypothesis, generally my example look like this

Where each function is just like this

I tested 5 variants here where I provided file path or URL as a source for image, for URL I used my own HA as a host and I also used some random cat image, all of URLs are accessible from my HA/NoneRed instance, file path is also correct

I will start with what works, this works (no issues):

https://t4.ftcdn.net/jpg/00/97/58/97/360_F_97589769_t45CqXyzjz0KXwoBZT9PRaWGHRk5hQqQ.jpg

But anything else is not, I used the same cat image and put it on my HA to test if the encoding is the issue, so none of below worked

https://192.168.1.10:8123/local/cat_img.jpg

/config/www/cat_img.jpg

https://192.168.1.10:8123/local/snapshot_camera.jpg

/config/www/snapshot_camera.jpg

Well all of those end with the same error message:
Caught exception in sender node:
Error: ETELEGRAM: 400 Bad Request: wrong file identifier/HTTP URL specified
when processing message: 
{"_msgid":"504f1dda4ed1b59e","payload":{"chatId":xxxx,"type":"photo","content":"https://192.168.1.10:8123/local/snapshot_camera.jpg","options":{"chat_id":xxxx,"photo":"https://192.168.1.10:8123/local/snapshot_camera.jpg"}},"topic":""}
...

OK, I checked one more thing, because now it appeared to me that all of those file are not being accessible from outside my network, I used an URL that points from the outside to one of my images and it worked

But this is strange because I swear it was working with local files without problems in the past, having my camera images accessible from the outside of the network is not ideal solution although I can implement some sort of "send and delete" flow

Sorry for delay. Well there are several ways to send photos:

content is a file name (that file is then uploaded to telegram from the node) (see https://github.com/windkh/node-red-contrib-telegrambot/blob/master/examples/sendphoto.json)
content is a buffer (you upload the content of a buffer instead of pointing to a file) (see https://github.com/windkh/node-red-contrib-telegrambot/blob/master/examples/sendphotobuffer.json)
content is a file_id (an ID of a file that was already uploaded to telegram before) (see https://core.telegram.org/bots/api#inputfile)

Reading the error message (wrong file identifier/HTTP URL specified) I assume that the content was an URL. In this case telegram tries to to the upload itself from the telegram server. In this case the file must be on a public reachable server. I guess that is not what you want to do.

Uploading a file from a local directory basically sends the content of the file to the telegram server. In this case the file format must be correct and the file must be readable from the node-red process (permissions and path must be correct and file must be written completely) --> this is what we must find out in a debugging session.

As an alternative you can load the file from the local hard disc into a buffer and use that method (see sendphotobuffer.json example I mentioned above)

windkh commented 1 month ago

@Bobo-amg I received a mail from you but I can not see the message here in the thread:

While continuing to debug this I am now getting the following:

Unhandled rejection TypeError: Converting circular structure to JSON --> starting at object with constructor 'IncomingMessage' | property 'res' -> object with constructor 'ServerResponse' --- property 'req' closes the circle at JSON.stringify () at TelegramOutNode.processError (/home/pi/.node-red/node_modules/node-red-contrib-telegrambot/te legrambot/99-telegrambot.js:2104:128) at /home/pi/.node-red/node_modules/node-red-contrib-telegrambot/telegrambot/99-telegrambot.js:22 57:46 at tryCatcher (/home/pi/.node-red/node_modules/bluebird/js/release/util.js:16:23) at Promise._settlePromiseFromHandler (/home/pi/.node-red/node_modules/bluebird/js/release/promis e.js:547:31) at Promise._settlePromise (/home/pi/.node-red/node_modules/bluebird/js/release/promise.js:604:18 ) at Promise._settlePromise0 (/home/pi/.node-red/node_modules/bluebird/js/release/promise.js:649:1 0) at Promise._settlePromises (/home/pi/.node-red/node_modules/bluebird/js/release/promise.js:725:1 8) at _drainQueueStep (/home/pi/.node-red/node_modules/bluebird/js/release/async.js:93:12) at _drainQueue (/home/pi/.node-red/node_modules/bluebird/js/release/async.js:86:9) at Async._drainQueues (/home/pi/.node-red/node_modules/bluebird/js/release/async.js:102:5) at Async.drainQueues [as _onImmediate] (/home/pi/.node-red/node_modules/bluebird/js/release/asyn c.js:15:14) at process.processImmediate (node:internal/timers:476:21)

was this deleted by you or is this still an issue?

Bobo-amg commented 4 weeks ago

Ok, in my case, the cause is either an incorrect mime type when I post via the python requests module, or it is the fact that saving a jpg file from a video stream results in a jpg which Telegram doesn't recognise. I have a workaround so case closed for me.

Thanks for the assistance.

windkh commented 4 weeks ago

@outlying what about you?

outlying commented 2 weeks ago

@windkh still the same case with all the stuff I mentioned, local URL / file path not working, external URL (mine or not) for same resource - no issues

windkh commented 2 weeks ago

@outlying please send me one of those images and that portion of the flow that you use for uploading

windkh / node-red-contrib-telegrambot

Error sending local image file as photo message #348