Closed merlea closed 6 years ago
Dear Antoine, what version of MDSplus are you using at the server side? A lot of work has been done a couple of years ago to reduce the amount of data handled by jScope (and therefore its memory footprint) but this requires an updated version of MDSplus at the server side. jScope communicates with the mdsip server in order to establish the optimal resampling factor, dynamically updated when zooming. However if the required set of funs and library functions at the server side is not available, jScope resorts to the old method, i.e. getting the whole set of waveform points.
Gabriele
On 24/05/2018 10:00, Antoine Merle wrote:
Hello,
I have noticed that there was quite a large increase in the memory footprint of jScope over the last years.
I am using as a test some overview scope for TCV. It has around 20 panels, 60 traces. The sampling of the traces is quite spread out and goes from 60 Hz to 100kHz. I use the MdsDataProvider class. I have compiled different versions using the same computer and the same JDK.
Using stable_release-7-0-X (the last of the 6-?-? cycle), the use of RAM is around 300MB (resident size). Using the latest stable_release or alpha_release, this goes up to 4000MB.
Is this something that you have seen as well? Do you know of any reason why this would have happened? Do you have any idea how to reduce the memory usage?
Thanks, Antoine.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319, or mute the thread https://github.com/notifications/unsubscribe-auth/AISySJl-6IE2uV-7HP0jXde_jwAgYkmZks5t1miQgaJpZM4ULvqA.
-- Gabriele Manduchi
Istituto Gas Ionizzati del CNR Consorzio RFX - Associazione EURATOM/ENEA sulla Fusione Corso Stati Uniti 4, 35127 Padova - Italy ph +39-049-829-5039/-5000 fax +39-049-8700718 mailto:gabriele.manduchi@igi.cnr.it, http://www.igi.cnr.it
Hi Gabriele,
Thank you for your swift response. We use 7-1-11 I think, so this is quite old already. We could try to arrange some test data on a newer version of the server.
I do have one question though, what about older versions of jScope (before 7), don't they also load the whole signals?
Antoine.
2018-05-24 10:10 GMT+02:00 GabrieleManduchi notifications@github.com:
Dear Antoine, what version of MDSplus are you using at the server side? A lot of work has been done a couple of years ago to reduce the amount of data handled by jScope (and therefore its memory footprint) but this requires an updated version of MDSplus at the server side. jScope communicates with the mdsip server in order to establish the optimal resampling factor, dynamically updated when zooming. However if the required set of funs and library functions at the server side is not available, jScope resorts to the old method, i.e. getting the whole set of waveform points.
Gabriele
On 24/05/2018 10:00, Antoine Merle wrote:
Hello,
I have noticed that there was quite a large increase in the memory footprint of jScope over the last years.
I am using as a test some overview scope for TCV. It has around 20 panels, 60 traces. The sampling of the traces is quite spread out and goes from 60 Hz to 100kHz. I use the MdsDataProvider class. I have compiled different versions using the same computer and the same JDK.
Using stable_release-7-0-X (the last of the 6-?-? cycle), the use of RAM is around 300MB (resident size). Using the latest stable_release or alpha_release, this goes up to 4000MB.
Is this something that you have seen as well? Do you know of any reason why this would have happened? Do you have any idea how to reduce the memory usage?
Thanks, Antoine.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319, or mute the thread https://github.com/notifications/unsubscribe- auth/AISySJl-6IE2uV-7HP0jXde_jwAgYkmZks5t1miQgaJpZM4ULvqA.
-- Gabriele Manduchi
Istituto Gas Ionizzati del CNR Consorzio RFX - Associazione EURATOM/ENEA sulla Fusione Corso Stati Uniti 4, 35127 Padova - Italy https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+Padova+-+Italy+%0D%0Aph+%2B39&entry=gmail&source=g ph +39-049-829-5039/-5000 fax +39-049-8700718 mailto:gabriele.manduchi@igi.cnr.it, http://www.igi.cnr.it
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391627845, or mute the thread https://github.com/notifications/unsubscribe-auth/ANe_XgbaHV7g0hREYatz1nCsF_Sfu_Zvks5t1msLgaJpZM4ULvqA .
Hi Antoine, yes, the older versions of jScope loaded the whole signal, however since the jScope code has deeply changed, it is hard for me to figure out the exact reason for the larger footprint (there are also several added features). There is for sure some added overhead for handling resampled signals, that may indeed cause larger memory requests if the signals are not resampled.
Gabriele
On 24/05/2018 10:32, Antoine Merle wrote:
Hi Gabriele,
Thank you for your swift response. We use 7-1-11 I think, so this is quite old already. We could try to arrange some test data on a newer version of the server.
I do have one question though, what about older versions of jScope (before 7), don't they also load the whole signals?
Antoine.
2018-05-24 10:10 GMT+02:00 GabrieleManduchi notifications@github.com:
Dear Antoine, what version of MDSplus are you using at the server side? A lot of work has been done a couple of years ago to reduce the amount of data handled by jScope (and therefore its memory footprint) but this requires an updated version of MDSplus at the server side. jScope communicates with the mdsip server in order to establish the optimal resampling factor, dynamically updated when zooming. However if the required set of funs and library functions at the server side is not available, jScope resorts to the old method, i.e. getting the whole set of waveform points.
Gabriele
On 24/05/2018 10:00, Antoine Merle wrote:
Hello,
I have noticed that there was quite a large increase in the memory footprint of jScope over the last years.
I am using as a test some overview scope for TCV. It has around 20 panels, 60 traces. The sampling of the traces is quite spread out and goes from 60 Hz to 100kHz. I use the MdsDataProvider class. I have compiled different versions using the same computer and the same JDK.
Using stable_release-7-0-X (the last of the 6-?-? cycle), the use of RAM is around 300MB (resident size). Using the latest stable_release or alpha_release, this goes up to 4000MB.
Is this something that you have seen as well? Do you know of any reason why this would have happened? Do you have any idea how to reduce the memory usage?
Thanks, Antoine.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319, or mute the thread https://github.com/notifications/unsubscribe- auth/AISySJl-6IE2uV-7HP0jXde_jwAgYkmZks5t1miQgaJpZM4ULvqA.
-- Gabriele Manduchi
Istituto Gas Ionizzati del CNR Consorzio RFX - Associazione EURATOM/ENEA sulla Fusione Corso Stati Uniti 4, 35127 Padova - Italy
https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+Padova+-+Italy+%0D%0Aph+%2B39&entry=gmail&source=g ph +39-049-829-5039/-5000 fax +39-049-8700718 mailto:gabriele.manduchi@igi.cnr.it, http://www.igi.cnr.it
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391627845, or mute the thread
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391633876, or mute the thread https://github.com/notifications/unsubscribe-auth/AISySJqm3f0mibu0dTlSRrRDjZQe6rqxks5t1nALgaJpZM4ULvqA.
-- Gabriele Manduchi
Istituto Gas Ionizzati del CNR Consorzio RFX - Associazione EURATOM/ENEA sulla Fusione Corso Stati Uniti 4, 35127 Padova - Italy ph +39-049-829-5039/-5000 fax +39-049-8700718 mailto:gabriele.manduchi@igi.cnr.it, http://www.igi.cnr.it
OK, I'll try to use a more recent version of the server and see if that improves the situation with the memory.
Do you know if it matters to have the data files already on the server or just re-direct to our older server would be enough?
Antoine.
2018-05-24 10:57 GMT+02:00 GabrieleManduchi notifications@github.com:
Hi Antoine, yes, the older versions of jScope loaded the whole signal, however since the jScope code has deeply changed, it is hard for me to figure out the exact reason for the larger footprint (there are also several added features). There is for sure some added overhead for handling resampled signals, that may indeed cause larger memory requests if the signals are not resampled.
Gabriele
On 24/05/2018 10:32, Antoine Merle wrote:
Hi Gabriele,
Thank you for your swift response. We use 7-1-11 I think, so this is quite old already. We could try to arrange some test data on a newer version of the server.
I do have one question though, what about older versions of jScope (before 7), don't they also load the whole signals?
Antoine.
2018-05-24 10:10 GMT+02:00 GabrieleManduchi notifications@github.com:
Dear Antoine, what version of MDSplus are you using at the server side? A lot of work has been done a couple of years ago to reduce the amount of data handled by jScope (and therefore its memory footprint) but this requires an updated version of MDSplus at the server side. jScope communicates with the mdsip server in order to establish the optimal resampling factor, dynamically updated when zooming. However if the required set of funs and library functions at the server side is not available, jScope resorts to the old method, i.e. getting the whole set of waveform points.
Gabriele
On 24/05/2018 10:00, Antoine Merle wrote:
Hello,
I have noticed that there was quite a large increase in the memory footprint of jScope over the last years.
I am using as a test some overview scope for TCV. It has around 20 panels, 60 traces. The sampling of the traces is quite spread out and goes from 60 Hz to 100kHz. I use the MdsDataProvider class. I have compiled different versions using the same computer and the same JDK.
Using stable_release-7-0-X (the last of the 6-?-? cycle), the use of RAM is around 300MB (resident size). Using the latest stable_release or alpha_release, this goes up to 4000MB.
Is this something that you have seen as well? Do you know of any reason why this would have happened? Do you have any idea how to reduce the memory usage?
Thanks, Antoine.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319, or mute the thread https://github.com/notifications/unsubscribe- auth/AISySJl-6IE2uV-7HP0jXde_jwAgYkmZks5t1miQgaJpZM4ULvqA.
-- Gabriele Manduchi
Istituto Gas Ionizzati del CNR Consorzio RFX - Associazione EURATOM/ENEA sulla Fusione Corso Stati Uniti 4, 35127 Padova - Italy https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+Padova+-+Italy&entry=gmail&source=g
https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+ Padova+-+Italy+%0D%0Aph+%2B39&entry=gmail&source=g ph +39-049-829-5039/-5000 fax +39-049-8700718 mailto:gabriele.manduchi@igi.cnr.it, http://www.igi.cnr.it
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391627845 , or mute the thread
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391633876,
or mute the thread https://github.com/notifications/unsubscribe-auth/ AISySJqm3f0mibu0dTlSRrRDjZQe6rqxks5t1nALgaJpZM4ULvqA.
-- Gabriele Manduchi
Istituto Gas Ionizzati del CNR Consorzio RFX - Associazione EURATOM/ENEA sulla Fusione Corso Stati Uniti 4, 35127 Padova - Italy https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+Padova+-+Italy+%0D%0Aph+%2B39&entry=gmail&source=g ph +39-049-829-5039/-5000 fax +39-049-8700718 mailto:gabriele.manduchi@igi.cnr.it, http://www.igi.cnr.it
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391641075, or mute the thread https://github.com/notifications/unsubscribe-auth/ANe_Xh8tZtft7eur933-tiFqIhPs0R8Sks5t1nXigaJpZM4ULvqA .
You can start a new server accessing data from the old one (in distributed client configuration). The new one will carry out dynamic resampling when requested by jScope
Gabriele
On 24/05/2018 11:15, Antoine Merle wrote:
OK, I'll try to use a more recent version of the server and see if that improves the situation with the memory.
Do you know if it matters to have the data files already on the server or just re-direct to our older server would be enough?
Antoine.
2018-05-24 10:57 GMT+02:00 GabrieleManduchi notifications@github.com:
Hi Antoine, yes, the older versions of jScope loaded the whole signal, however since the jScope code has deeply changed, it is hard for me to figure out the exact reason for the larger footprint (there are also several added features). There is for sure some added overhead for handling resampled signals, that may indeed cause larger memory requests if the signals are not resampled.
Gabriele
On 24/05/2018 10:32, Antoine Merle wrote:
Hi Gabriele,
Thank you for your swift response. We use 7-1-11 I think, so this is quite old already. We could try to arrange some test data on a newer version of the server.
I do have one question though, what about older versions of jScope (before 7), don't they also load the whole signals?
Antoine.
2018-05-24 10:10 GMT+02:00 GabrieleManduchi notifications@github.com:
Dear Antoine, what version of MDSplus are you using at the server side? A lot of work has been done a couple of years ago to reduce the amount of data handled by jScope (and therefore its memory footprint) but this requires an updated version of MDSplus at the server side. jScope communicates with the mdsip server in order to establish the optimal resampling factor, dynamically updated when zooming. However if the required set of funs and library functions at the server side is not available, jScope resorts to the old method, i.e. getting the whole set of waveform points.
Gabriele
On 24/05/2018 10:00, Antoine Merle wrote:
Hello,
I have noticed that there was quite a large increase in the memory footprint of jScope over the last years.
I am using as a test some overview scope for TCV. It has around 20 panels, 60 traces. The sampling of the traces is quite spread out and goes from 60 Hz to 100kHz. I use the MdsDataProvider class. I have compiled different versions using the same computer and the same JDK.
Using stable_release-7-0-X (the last of the 6-?-? cycle), the use of RAM is around 300MB (resident size). Using the latest stable_release or alpha_release, this goes up to 4000MB.
Is this something that you have seen as well? Do you know of any reason why this would have happened? Do you have any idea how to reduce the memory usage?
Thanks, Antoine.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319, or mute the thread https://github.com/notifications/unsubscribe- auth/AISySJl-6IE2uV-7HP0jXde_jwAgYkmZks5t1miQgaJpZM4ULvqA.
-- Gabriele Manduchi
Istituto Gas Ionizzati del CNR Consorzio RFX - Associazione EURATOM/ENEA sulla Fusione Corso Stati Uniti 4, 35127 Padova - Italy
https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+Padova+-+Italy&entry=gmail&source=g
https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+ Padova+-+Italy+%0D%0Aph+%2B39&entry=gmail&source=g ph +39-049-829-5039/-5000 fax +39-049-8700718 mailto:gabriele.manduchi@igi.cnr.it, http://www.igi.cnr.it
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub
<https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391627845 , or mute the thread
— You are receiving this because you commented. Reply to this email directly, view it on GitHub
https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391633876,
or mute the thread https://github.com/notifications/unsubscribe-auth/ AISySJqm3f0mibu0dTlSRrRDjZQe6rqxks5t1nALgaJpZM4ULvqA.
-- Gabriele Manduchi
Istituto Gas Ionizzati del CNR Consorzio RFX - Associazione EURATOM/ENEA sulla Fusione Corso Stati Uniti 4, 35127 Padova - Italy
https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+Padova+-+Italy+%0D%0Aph+%2B39&entry=gmail&source=g ph +39-049-829-5039/-5000 fax +39-049-8700718 mailto:gabriele.manduchi@igi.cnr.it, http://www.igi.cnr.it
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391641075, or mute the thread
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391646447, or mute the thread https://github.com/notifications/unsubscribe-auth/AISySBz4LluNwp2PFGNiZL9sGfjlXGF-ks5t1no-gaJpZM4ULvqA.
-- Gabriele Manduchi
Istituto Gas Ionizzati del CNR Consorzio RFX - Associazione EURATOM/ENEA sulla Fusione Corso Stati Uniti 4, 35127 Padova - Italy ph +39-049-829-5039/-5000 fax +39-049-8700718 mailto:gabriele.manduchi@igi.cnr.it, http://www.igi.cnr.it
Hi Gabriele,
From what I can tell, using a more recent version of the server improves the situation very much. It looks like the memory usage is even less than the older versions (about 200MB). This probably means that it is the compatibility layer in jScope that is goggling the memory. Thank you for the tip.
So where are the server-side components of jScope. Are they only in mdsmisc? I am asking because it might be easier for us to update only part of the mdsplus installation on the general TCV server.
Thanks again, Antoine.
2018-05-24 11:34 GMT+02:00 GabrieleManduchi notifications@github.com:
You can start a new server accessing data from the old one (in distributed client configuration). The new one will carry out dynamic resampling when requested by jScope
Gabriele
On 24/05/2018 11:15, Antoine Merle wrote:
OK, I'll try to use a more recent version of the server and see if that improves the situation with the memory.
Do you know if it matters to have the data files already on the server or just re-direct to our older server would be enough?
Antoine.
2018-05-24 10:57 GMT+02:00 GabrieleManduchi notifications@github.com:
Hi Antoine, yes, the older versions of jScope loaded the whole signal, however since the jScope code has deeply changed, it is hard for me to figure out the exact reason for the larger footprint (there are also several added features). There is for sure some added overhead for handling resampled signals, that may indeed cause larger memory requests if the signals are not resampled.
Gabriele
On 24/05/2018 10:32, Antoine Merle wrote:
Hi Gabriele,
Thank you for your swift response. We use 7-1-11 I think, so this is quite old already. We could try to arrange some test data on a newer version of the server.
I do have one question though, what about older versions of jScope (before 7), don't they also load the whole signals?
Antoine.
2018-05-24 10:10 GMT+02:00 GabrieleManduchi notifications@github.com:
Dear Antoine, what version of MDSplus are you using at the server side? A lot of work has been done a couple of years ago to reduce the amount of data handled by jScope (and therefore its memory footprint) but this requires an updated version of MDSplus at the server side. jScope communicates with the mdsip server in order to establish the optimal resampling factor, dynamically updated when zooming. However if the required set of funs and library functions at the server side is not available, jScope resorts to the old method, i.e. getting the whole set of waveform points.
Gabriele
On 24/05/2018 10:00, Antoine Merle wrote:
Hello,
I have noticed that there was quite a large increase in the memory footprint of jScope over the last years.
I am using as a test some overview scope for TCV. It has around 20 panels, 60 traces. The sampling of the traces is quite spread out and goes from 60 Hz to 100kHz. I use the MdsDataProvider class. I have compiled different versions using the same computer and the same JDK.
Using stable_release-7-0-X (the last of the 6-?-? cycle), the use of RAM is around 300MB (resident size). Using the latest stable_release or alpha_release, this goes up to 4000MB.
Is this something that you have seen as well? Do you know of any reason why this would have happened? Do you have any idea how to reduce the memory usage?
Thanks, Antoine.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319, or mute the thread https://github.com/notifications/unsubscribe- auth/AISySJl-6IE2uV-7HP0jXde_jwAgYkmZks5t1miQgaJpZM4ULvqA.
-- Gabriele Manduchi
Istituto Gas Ionizzati del CNR Consorzio RFX - Associazione EURATOM/ENEA sulla Fusione Corso Stati Uniti 4, 35127 Padova - Italy https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+Padova+-+Italy&entry=gmail&source=g
https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+ Padova+-+Italy&entry=gmail&source=g
https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+ Padova+-+Italy+%0D%0Aph+%2B39&entry=gmail&source=g ph +39-049-829-5039/-5000 fax +39-049-8700718 mailto:gabriele.manduchi@igi.cnr.it, http://www.igi.cnr.it
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub
<https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391627845 , or mute the thread
— You are receiving this because you commented. Reply to this email directly, view it on GitHub
https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391633876,
or mute the thread https://github.com/notifications/unsubscribe-auth/ AISySJqm3f0mibu0dTlSRrRDjZQe6rqxks5t1nALgaJpZM4ULvqA.
-- Gabriele Manduchi
Istituto Gas Ionizzati del CNR Consorzio RFX - Associazione EURATOM/ENEA sulla Fusione Corso Stati Uniti 4, 35127 Padova - Italy https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+Padova+-+Italy&entry=gmail&source=g
https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+ Padova+-+Italy+%0D%0Aph+%2B39&entry=gmail&source=g ph +39-049-829-5039/-5000 fax +39-049-8700718 mailto:gabriele.manduchi@igi.cnr.it, http://www.igi.cnr.it
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391641075 , or mute the thread
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391646447,
or mute the thread https://github.com/notifications/unsubscribe-auth/ AISySBz4LluNwp2PFGNiZL9sGfjlXGF-ks5t1no-gaJpZM4ULvqA.
-- Gabriele Manduchi
Istituto Gas Ionizzati del CNR Consorzio RFX - Associazione EURATOM/ENEA sulla Fusione Corso Stati Uniti 4, 35127 Padova - Italy https://maps.google.com/?q=Corso+Stati+Uniti+4,+35127+Padova+-+Italy+%0D%0Aph+%2B39&entry=gmail&source=g ph +39-049-829-5039/-5000 fax +39-049-8700718 mailto:gabriele.manduchi@igi.cnr.it, http://www.igi.cnr.it
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/1319#issuecomment-391651962, or mute the thread https://github.com/notifications/unsubscribe-auth/ANe_Xm3R5ptPhdHHadBf87W0EcnKXzmIks5t1n6wgaJpZM4ULvqA .
Closing this issue. Antoine.
Hello,
I have noticed that there was quite a large increase in the memory footprint of jScope over the last years.
I am using as a test some overview scope for TCV. It has around 20 panels, 60 traces. The sampling of the traces is quite spread out and goes from 60 Hz to 100kHz. I use the MdsDataProvider class. I have compiled different versions using the same computer and the same JDK.
Using stable_release-7-0-X (the last of the 6-?-? cycle), the use of RAM is around 300MB (resident size). Using the latest stable_release or alpha_release, this goes up to 4000MB.
Is this something that you have seen as well? Do you know of any reason why this would have happened? Do you have any idea how to reduce the memory usage?
Thanks, Antoine.