search

Z-Hub / Z-Push

Home of Z-Push
https://z-push.org
GNU Affero General Public License v3.0
97 stars 47 forks source link

Unable to download or open attachments on android #8

Closed matidau closed 9 months ago

matidau commented 1 year ago

Server (please complete the following information):

Smartphone (please complete the following information):

Additional context

Originally posted by @draga79 in https://github.com/Z-Hub/Z-Push/issues/5#issuecomment-1552879953

I have noticed another problem, but I believe it is more related to the Zimbra Backend. On my iPhone, everything appears to be functioning correctly. However, on Android (using the Nine mail client), I am unable to download or open attachments. Instead, I receive the following error message:

"Fatal error: /z-push/backend/zimbra/zimbraHttpStreamWrapper.php:111 - Uncaught TypeError: fread(): Argument #1 ($stream) must be of type resource, bool given in /z-push/backend/zimbra/zimbraHttpStreamWrapper.php:111"

Should I report this issue to the Zimbra Backend developers?

Thank you, Stefano

liverpoolfc-fan

Are you using the latest codebase for z-push from here, or using the fix posted on the Z-Push Zimbra Backend sourceforge ticket?

If code from here then I believe you shouldn't experience that problem.

If you used my old fix from the Sourceforge site - please see a correction I just made to it. https://sourceforge.net/p/zimbrabackend/support-requests/185/#934b

Only the STREAMER_TYPE_STREAM_ASPLAIN type should use the resource converter. Using it on STREAMER_TYPE_STREAM_BASE64 broke downloads of attachments. Uncomment the original line for that and delete the three lines that were added. -OR- As Z-Push now has a new maintainer, you could choose to throw away these two unofficial mods and download the latest Z-Push code from https://github.com/Z-Hub/Z-Push

draga79

Yes, it is happening with the codebase pulled from here today via "git clone".

matidau commented 1 year ago

Hi Stefano @draga79,

It would be good to set the log level to debug if you haven't already and to provide a few more lines above this one

Fatal error: /z-push/backend/zimbra/zimbraHttpStreamWrapper.php:111 - Uncaught TypeError: fread(): Argument https://github.com/Z-Hub/Z-Push/pull/1 ($stream) must be of type resource, bool given in /z-push/backend/zimbra/zimbraHttpStreamWrapper.php:111

Instructions for setting the log level are here on the wiki, please use LOGLEVEL_DEBUG instead of LOGLEVEL_WBXML.

Also just a couple of questions when testing between the iPhone and Android:

While testing for 8.2 compatibility on Android, I had some trouble opening in Nine mail with some imported items (though to different backends: imap, carddav, caldav), whereas the Gmail app didn't have these problems. Nine didn't seem to have problems with new non-imported items. I also found that when an error in z-push was encountered, I would sometimes have to remove the account from Nine mail and add it back in again so that it could start to work again. This leads me to a couple of more questions:

Cheers, Mat

draga79 commented 1 year ago

Hello @matidau ,

It would be good to set the log level to debug if you haven't already and to provide a few more lines above this one

I've set it to debug but no difference - the same error gets logged, nothing before of after that. Just that.

Also just a couple of questions when testing between the iPhone and Android: are you using the same mail account between both devices? are you opening the same attachments/emails?

Yes, same account and same email/attachment. I'm testing using: iOS 16.5 on iPhone 13 Pro Max and iPhone 8 with 16.4.1 (both with Apple Mail and both work), Android 13 on Pixel 7 and a Mi 9T Pro - with Android 13 via the EvolutionX rom (with Nine and Gmail)

have you tested with Gmail on Android?

I did, same error

are these newly created items or are they imported in to Zimbra?

They're e-mails I've received. This Zimbra server has been online since 2015, so all the contents are newly created.

have you had to remove the account from Nine mail and add it back in?

Yes, I did. No difference, everything is right with normal e-mails (contacts, calendar events, etc) but cannot download or view e-mail attachments on the Android devices, both with Nine and Gmail.

Thank you, Stefano

liverpoolfc-fan commented 1 year ago

Hi @matidau,

I believe the issue is where/how you have tried to implement the create resource from string. It is being applied to all date that is in a string. Unfortunately, WBXML base64 data is just a string that has a particular character set - but to PHP is is just a string.

I believe my approach on the sourceforge ticket referenced in my previous posting is the correct approach to solving this. It looks at the WBXML data type, and only calls the function for STREAMER_TYPE_STREAM_ASPLAIN

Please take a look at the approach. https://sourceforge.net/p/zimbrabackend/support-requests/185/#934b

Additionally, I think for the purpose of limiting the memory used by the process the maxmemory parameter should be used when creating the stream resource. I arbitrarily used a max of 5MB. This could potentially be added to the config - but it is a limit after which it will use a temp file to store the stream data to avoid exhausting PHP's memory

    $fiveMBs = 5 * 1024 * 1024;
    $stream = fopen('php://temp/maxmemory:$fiveMBs', 'r+');
liverpoolfc-fan commented 1 year ago

Hi @matidau,

I hadn't thought about it at the time of the sourceforge ticket - I was just looking for an effective solution. But, now that this github is re-opened, perhaps utils.php would be the best file for the string to stream function. What do you think?

matidau commented 1 year ago

I'm going to investigate a bit more on this one. Imap backend is working with attachments on Android. Not sure if this is due to configuration or the backend. Going to launch another VM to test this further.

liverpoolfc-fan commented 1 year ago

I believe IMAP backend is self-contained. As far as I am aware it is not built as an extension to diffbackend. This could potentially be a source of different behaviour.

matidau commented 1 year ago

It extends BackendDiff https://github.com/Z-Hub/Z-Push/blob/6458636d2964d2b7398ae356524938a41d1ba8c6/src/backend/imap/imap.php#L40

But that isn't to say that the attachments work in the same way or that it needs to be fixed in the individual backends. I'm not quite setup with the various environments for testing yet, but did you want to submit a PR for this?

matidau commented 1 year ago

Hi Stefano @draga79

Just coming back to this one, to see if you have any updates.

I've simulated what you are experiencing, and if you would like to try modifying line 111 of z-push/backend/zimbra/zimbraHttpStreamWrapper.php

from:

        $data = fread($this->zimbrahttpstream, $len);

to:

        if($this->zimbrahttpstream)
            $data = fread($this->zimbrahttpstream, $len);

and see if this alleviates the error message.

After working on replicating this I would like to get your thoughts on your set up and the files that you are working with.

While testing with the GMail App to replicate this, the UI behaviour had some flickering of attachment previews, i.e. maybe it was retrying to fetch them, or struck out images. On your iPhone have you noticed a delay or interesting behaviour such as this in getting the attachments?

Are you running z-push on a separate VM, instance, or container than you Zimbra installation? Is there perhaps a Networking problem that is causing this to occur? I'm new to Zimbra so uncertain if this is the common way for Z-Push and Zimbra to be set up in Production. Just with the other issue for intermittent contacts, leads me to think that there is a problem between Z-Push and Zimbra talking as opposed to just Attachments, but maybe the attachments are more obvious as they are a larger contents.

liverpoolfc-fan commented 1 year ago

You will never see an issue on iDevices because they request the complete MIME message (including attachments) from the server before displaying it.

Android devices request either the HTML or Text content of the message, and the list of atttachmnts. Only if the user clicks on an attachment is a request sent to the server to fetch the content of the attachment.

draga79 commented 1 year ago

I haven't had time to try again, yet, but will do it in some days. My z-push installation has been performed on an OpenBSD VM in the same lan as the Zimbra server. I'll report back as soon as I'll have performed some tests.

liverpoolfc-fan commented 1 year ago

@draga79 @matidau

Can you please detail the versions of Z-Push, Zimbra and the Z-Push Zimbra Backend you are using. I have created a test install this morning and run some tests and I cannot reproduce your issue.

I have created a new test Z-Push install (cloned the latest from here - develop-2.7.x-6-g7b3432fa), and installed the lastest Z-Push Zimbra Backend (Download Release 71 package from Sourceforge, then replaced zimbra.php with not-yet-packaged version 71.3 from the Sourceforge Code Repository. I am running this against zimbra 8.8.15-latest-patch

I have tested with the Samsung Email client and the Nine client. No issues with either.

matidau commented 1 year ago

@liverpoolfc-fan

Just to clarify I was not able to replicate without simulating a failure of the attachment retrieval.

I tested with the same setup as you listed and separately with 2.7.0 Z-Push and r71 Zimbra Backend.

Without simulating a failure it works perfectly. I used a similar method to simulate the failure for #5 , though that one that could also happen in other backends.

I suspect that there is a Network or environmental issue that @draga79 is experiencing that is causing both of these issues.

matidau commented 9 months ago

Hi @draga79

Not sure if you have had a chance to investigate or test this further.

Going to close this off, I'm still of the belief that you were experiencing and issue outside of Z-Push directly.

Happy to be proven wrong, please respond to this if that is the case.

Cheers, Mat

draga79 commented 9 months ago

Hello @matidau
Sorry for not replying earlier - I replaced the OpenBSD operating system with FreeBSD (setting up z-push in a dedicated jail) and the issue hasn't arisen since. I wouldn't rule out the possibility of an issue related to OpenBSD's hardening (and its corresponding PHP compiled on it), but since I need a reliable system, I opted for a more conservative setup. I might conduct some tests again in the future.

Thanks again for your support!