Improve import of large OPML files

[Kaligule]

File can be found here: https://owncloud.tillmail.de/index.php/s/XCaGw8dvJjJVoYK I tried the one with the settings included. When the file is upladed, I am informed that the feeds are getting imported now. Unfortunatelly, no feeds are imported and I cannot import the other file (without the settings). I suggest:

• add option to terminate the import process (in case it doesn't work)
• add error message when the opml file cannot be parsed

[amatriain]

I'm looking into it and I think your import eventually finished successfully. Can you check again if the import process finished? You should see a notification once logged in, and you should have received an email.

Unfortunately importing subscriptions is a slow process, specially if there are many feeds in the OPML that no other user is subscribed to yet. Also if there is a large backlog of users importing their OPMLs it may take some time until the process starts.

I've tuned queue priorities so that the import OPML process starts faster, having higher priority than other background jobs that may be competing in the queue. Hopefully this improves user experience.

Your OPML files (thanks for sharing them btw) are fine, Feedbunch parses them correctly. I've imported them myself without issue.

About your suggestions:

• if the OPML file cannot be parsed (it is not valid xml, it contains no subscriptions etc) the user will be notified with an error message and he will be given the option to upload another file. This is already implemented.
• currently a running import process cannot be terminated. I like the idea of having a "cancel" button however. I'll work on it.

Thanks!

[Kaligule]

Yea, you were right. I did finish, but took very long. There might be an other solution. I assume the parsing of the file is quick (I don't see why it should not be), but importing the feeds itself (going to the website, loading the articles and other media, doing database things, looking up if this feed has been subscribed by some other users) is slow.

So perhaps, instead doing all of this at once, you could parse the feednames and directory structure from the opml and add it immediatelly, with dummy items for the feeds. After that, you do the hard work in the background and add the feeds one after another. To indicate that a feed is not ready yet you perhaps could simply outgrey his name or display some progressbar over it.

This would prevent the user from thinking "It tooks longer then 10 seconds, it must be a bug!". He/She would get feedback almost immediately and could do read some feeds while the other feeds are imported.

This is just me doodling around ideas about user interfaces. I never have written a single line of ruby code and don't know how much effort this would be (or weather it is possible),

[amatriain]

What you suggest is certainly possible from a technical point of view. However having individual progress indicators for each feed in the OPML would be more complex than the current implementation, which shows a single progress bar for the whole import process.

Also I'm not sure this would fix the issue of users thinking there's a bug if the process takes a long time to start. I think that if a lot of feeds immediately appear in the UI but all of them show "in progres..." for a long time a user may think a bug has happened, same as if the global progress bar stays at 0 for a long time.

I'm not convinced the added complexity is worth it. I think it's better to improve user experience by making import jobs start faster (the recent changes in queue priorities should help with that) and to make clear to the user that the import process is long and to be patient (the messages shown to the user could probably be improved).

[Kaligule]

I think you are right.

[amatriain]

Thanks for your feedback. Closing this for now.