eclipse-amalgam-bot
commented
2 years ago cedric.brun commented on 2013-07-17
Thanks for taking the time to do this bug report.
After a quick analysis : the code was giving the illusion it was done async but in the end was not because createControls() would trigger the retrievial of the catalog anyway.
I'm looking into it and will get back to you with my progress. I'll try to handle the "cancel" correctly though as I don't have control on the Resource loading per say I'm not sure I'll be able to cancel in the middle of the download...
On the other hand the catalog right now is an XMI serialization, I might also have a look on the binary serialization provided by emf but the catalog is a 19k file so I guess your problem is more about latency (or the verification of the availability which will be more network intensive...)
joost.van.pinxten commented on 2013-07-17
(In reply to comment #1)
Thanks for taking the time to do this bug report.
After a quick analysis : the code was giving the illusion it was done async but in the end was not because createControls() would trigger the retrievial of the catalog anyway.
I'm looking into it and will get back to you with my progress. I'll try to handle the "cancel" correctly though as I don't have control on the Resource loading per say I'm not sure I'll be able to cancel in the middle of the download...
On the other hand the catalog right now is an XMI serialization, I might also have a look on the binary serialization provided by emf but the catalog is a 19k file so I guess your problem is more about latency (or the verification of the availability which will be more network intensive...)
Cheers for taking it seriously. Somehow, the connection to the Eclipse servers is an iffy one; I've tried from several internet connections (University of Eindhoven, University of York, several home broadband connections) and have been able to reproduce this on all lines. Overall, the network connections are good; i.e. > 20 megabit download, > 1 megabit upload and < 15 ms latency to e.g. Google, but Eclipse.org is somehow always above 100ms.
Unfortunately, the behaviour I've described in the bug also exists in the update site mechanism (which is sometimes blazing fast and other times painstakingly slow). Anyway, the update site can at least be (somewhat) properly cancelled, which is the reason that I've submitted this bug here.
I'm actually unsure whether or not this has anything to do with latency or bandwidth, perhaps it is some kind of (unauthorized?) DNS lookup that is causing the delay (for over 1 minute though).
cedric.brun commented on 2013-07-17
For the record, on linux I simulate a fairly high network latency with this :
sudo tc qdisc add dev eth2 root netem delay 1000ms
And then I can reproduce and test the fact that the UI is very painful (because it freezes the UI) under those conditions.
cedric.brun commented on 2013-07-17
After digging further :
- the original catalog loading is happening in the UI thread. We should be able to overcome that by using the progress service and a progress dialog. Then I need to see how I can really handle the cancel event
- during the createContent of the viewer org.eclipse.amalgam.discovery.ui.viewer.DiscoveryViewer.ConnectorDescriptorItemUi.ConnectorDescriptorItemUi(DiscoveryViewer, InstallableComponent, Composite, Color) are created and are triggerring the HTTP resolution of the icons to set them as SWTLabels. Here again I should find a way to do that in an async way.
cedric.brun commented on 2013-07-17
Move the loading of the catalog out of the UI thread and provide the ability to cancel. This is done through CancellableInputStream which will throw an exception during read operations if the monitor has been cancelled in the meantime.
Makes the icons retrieval async using the Job API, a placeholder icon being used in the meantime.
Caches the images so that they are not getting downloaded again when using a filter in the wizard.
Build https://hudson.eclipse.org/hudson/job/amalgam-1.2.0/277/ contains the fixes. That said, you might not be able to install it in your modeling package and might have to wait for the next Kepler milestone preparing Kepler SR1. If you want to test in the meantime (and I'd really like to have your feedback) please use any other Eclipse distribution (Standard for instance) and install the features you'll find in the https://hudson.eclipse.org/hudson/job/amalgam-1.2.0/277/artifact/features/org.eclipse.amalgam.update/target/org.eclipse.amalgam.update.zip update site.
Thanks again for your report. Feel free to comment or reopen it if needed.
cedric.brun commented on 2013-07-17
fixed with build https://hudson.eclipse.org/hudson/job/amalgam-1.2.0/277/
joost.van.pinxten commented on 2013-07-17
(In reply to comment #5)
http://git.eclipse.org/c/amalgam/org.eclipse.amalgam.git/commit/ ?id=05e3c7319b996602b805a07f9e9136365e4053a5
Move the loading of the catalog out of the UI thread and provide the ability to cancel. This is done through CancellableInputStream which will throw an exception during read operations if the monitor has been cancelled in the meantime.
http://git.eclipse.org/c/amalgam/org.eclipse.amalgam.git/commit/ ?id=638ddbbd03dce44f4bec52e8ebd51a336efd7b67
Makes the icons retrieval async using the Job API, a placeholder icon being used in the meantime.
http://git.eclipse.org/c/amalgam/org.eclipse.amalgam.git/commit/ ?id=b67ff672a285a3d4cf2130cd737ac0a38a195ce4
Caches the images so that they are not getting downloaded again when using a filter in the wizard.
Build https://hudson.eclipse.org/hudson/job/amalgam-1.2.0/277/ contains the fixes. That said, you might not be able to install it in your modeling package and might have to wait for the next Kepler milestone preparing Kepler SR1. If you want to test in the meantime (and I'd really like to have your feedback) please use any other Eclipse distribution (Standard for instance) and install the features you'll find in the https://hudson.eclipse.org/hudson/job/amalgam-1.2.0/277/artifact/features/ org.eclipse.amalgam.update/target/org.eclipse.amalgam.update.zip update site.
Thanks again for your report. Feel free to comment or reopen it if needed.
Thanks Cedric, for picking this up. I think it has bugged quite some users already, but I guess none of them took the time to report it (or knew where to report it).
I'll have a look at it, probably tomorrow. It's on my TODO list now ;-)
joost.van.pinxten commented on 2013-07-18
(In reply to comment #7)
Thanks again for your report. Feel free to comment or reopen it if needed.
Thanks Cedric, for picking this up. I think it has bugged quite some users already, but I guess none of them took the time to report it (or knew where to report it).
I'll have a look at it, probably tomorrow. It's on my TODO list now ;-)
Alright, that's a whole bunch better already! However, cancelling the progress dialog does seem to take quite some time (i.e. sometimes > 10 seconds).
I've installed Wireshark to see what happens in the background. The following HTTP requests seem to fail (404's) but take no longer than 200ms to respond: http://download.eclipse.org/egf/updates/kepler/official/content.xml http://download.eclipse.org/emf-store/releases/latest/site.xml http://mirror.anl.gov/pub/repastsimphony/compositeContent.jar
I haven't used Wireshark before, so don't know exactly what the entries all mean and therefore may not be able to provide the right information.
The Amalgam dialog basically gets stuck on 'Checking update sites availability' for a long time. Some of the icons (e.g. Acceleo, Jet, Operational QVT, Extended Editing Framework and some others) don't load, and only show a small red square (I think 4x4px).
Clicking on the license icon of a row (such as Xtext), which are located at the far right (next to "by Eclipse.org EPL", also stops the UI from responding (rather briefly). And when it is responding again, nothing is shown; the list IS refreshed.
I don't know if this is specific enough to work with. If I can provide you with more information, please feel free to ask me.
Cheers!
cedric.brun commented on 2013-08-01
I tried with a delay for network transferts of 5 sec and could reproduce the fact that cancel will "wait" for a bit more than 10 sec to be effective.
Debugging the code, we're deep into Java's URL and HTTPConnections, DNS lookups and Sockets being initialized to transmit data.
I have no idea how I could plug my cancellation check so down in the stack. Right now the cancel is checked when the inputstream is being read (see http://git.eclipse.org/c/amalgam/org.eclipse.amalgam.git/tree/plugins/org.eclipse.amalgam.discovery.core/src/org/eclipse/amalgam/discovery/core/CancellableInputStream.java ) but all the connection initiation is done by the underlying frameworks and that includes the dns lookup.
Icons not loading means some HTTP requests to them are failling, either because of a timeout or maybe a failed redirect (?). The only thing I could try would be to limit the number of http request done at the same time (right now the code is asking for all of them, at once) but I'm not sure it would fix the problem you experienced.
I used wireshack / ehteral too : some HTTP are returning 302 , more especially urls related to emfstore (http://download.eclipse.org/emf-store/releases/latest which is not available anymore ) and Agent Modeling Platform (but it ends up finding the resources once redirected).
I just fixed the emfstore URL in the catalog. Is that helping in your case ?
When using a slow/unstable internet connection (e.g. on a wireless connection, or when the Eclipse servers are heavily loaded), it is very unfortunate to use the Modeling components interface that comes with the Modeling Tools distribution of Eclipse. It means the whole Eclipse interface will hang/stop responding until the elements are loaded. As there is no way to see progress or even to cancel the request, I really think this is NOT a good design of that interface at all.
This problem is most likely due to a synchronous http-request in an UI thread. There are (as far as I can tell) two places where the interface will hang; one on opening the window (I think just loading the list) and the second when the window is opened (and you can (try to) click the little red square to stop the request). The interface already hangs BEFORE any of the modeling plugins can be selected.
I've downloaded a fresh Kepler 4.3.0 Modeling Tools distribution from http://www.eclipse.org/downloads/packages/eclipse-modeling-tools/keplerr (no, the double r of kepler is not my typo)
The steps to reproduce this are:
At this point, the interface will hang, cannot be properly cancelled and that's just rubbish and will most probably drive new users from using Eclipse. (It's even worse than the Software Update implementation in this respect...)
Please fix this issue, as I've noticed that when it DOES work it's a great addition to the Eclipse plugin installation process.
Please see also bug #412831 for my original bug post (basically, I didn't know what product I had to post this bug in).
🆔 ECLIPSE-413097👷 joost.van.pinxten📅 2013-07-16