search

sardana-org / sardana

Moved to GitLab: https://gitlab.com/sardana-org/sardana
39 stars 52 forks source link

Support python3 in sardana #1089

Closed cpascual closed 5 years ago

cpascual commented 5 years ago

We all know it is coming... https://pythonclock.org/

... so here is this issue as a reminder ;)

Refer to https://github.com/taurus-org/taurus/pull/703 for ideas.

reszelaz commented 5 years ago

I propose the following roadmap:

reszelaz commented 5 years ago

This is a quick poll: do you see it feasible on your institutes to migrate Sardana directly to Python 3 without compatibility with Python 2?

Let's say, from the next milestone (Jul19 or Jan20), the develop and master branches will be Python 3 only compatible. We could of course add a Python 2 support branch for very hot fixes. This decission will require you to migrate your plugins (macros, controllers, etc) to Python 3.

As said, this is just a quick poll, more to hear strong "No" if any... Of course this topic will be deeply analyzed (also during the Sardana Workshop next week in Hamburg).

cpascual commented 5 years ago

This of course will require you to migrate your plugins (macros, controllers, etc) to Python 3.

One note: in order to facilitate the transition, we could try to implement an auto-convert-on-import mechanism (based on the past module) that may facilitate using py2 macros/controllers with a py3 only sardana. I haven't got experience with it, so I cannot say for sure it will work for all our macros/controllers, but we can do some tests.

teresanunez commented 5 years ago

Hi, it should not be a problem for DESY, we have to change in any case. I could also contribute in the transition.

guifrecuni commented 5 years ago

Hello, Id' vote for migration to Python 3 without compatibility with Python2. Nonetheless, I think it is worth to evaluate how much complexity implies that compatibility, because making it "transparent" wrt macros/controllers code may imply too much burden, I guess. Forcing the migration to Python3 may also have some advantages, since specific focus to each functionality will allow some people to rethink/redesign or reduce technical debt. Finally, for the transition I see this sceneario that would help everyone: Python3 Pool + MS TANGO devices providing a "door3" to test new macros and a Python2 MS TANGO device connected to Python3 Pool(s) devices with a door. With this setup users may be able to test their macros with the same hardware environment and compare functionality.

daneos commented 5 years ago

I think supporting both Py2 and Py3 would be too much overhead, since sooner or later we all will need to make all of our code Python3-compatible. On the other hand, migrating all plugins at once seems to be rather massive task, so if the solution proposed by @cpascual would work, that'd be just perfect.

In any case, I support the idea of Python3-only Sardana, but I think we should mark the last Py2 release as long time support and still provide bugfixes for it for some extended time.

dschick commented 5 years ago

I have no problems with Py3 only. I would heavily appriciate it.

reszelaz commented 5 years ago

From the above answers and a quick poll during the last Sardana Workshop it looks like we all agree to go directly to Python 3.

Another thing that we need to agree is to which exact version of Python 3? If I remember correctly these are the default Python 3 versions that we run in production

From the above options, I have the feeling that we could go directly towards Python 3.5 (MaxIV and Solaris plan to upgrade soon to CentOS 8?) Alternativelly going towards Python 3.7 would require running Sardana in production with non default version of Python to your OS, for example, using it within a conda environment. Are you considering this as an option or you would prefer to stick to the default OS version?

Regarding the Sardana code base migration to Python 3. What do you think about the following plan?

  1. Make the Jul19 release (including SEP2) - the last Python 2 compatible release, ideally within two weekes.
  2. Create support-2.x branch from the develop branch just after the Jul19 release.
    • the idea of this branch is to eventually backport important bug fixes from the future develop branch (Python 3)
  3. Automatically futurize the develop branch and manually fix eventual problems.
    • this change will require dropping support to Taurus 3 at the same time
    • when testing and fixing problems we will focus first on the MacroServer (necessary by the ESRF before August this year)
    • also we may drop compatibility with the features deprecated in the previus releases hence the next Sardana release Jan20 will be most probably a major version release.

Of course the most convenient to all of us is that as soon as we have migrated Sardana code base to Python 3 we all start using it in production. This is in order to not have to switch context between development of new features and giving support (debugging problems in production) during too much time. Also the eventual bug fixes won't need to be backported to the support-2.x which would save us a lot of time. But as mentioned in the previous comments this will require migrating all the plugins to Python 3 as well...

We at Alba will make a strong focus on migrating our beamlines to this new code base as quickly as possible. We start already today by making an inventory of our plugins and their dependencies. We plan to start migraging our controllers to Python 3 ASAP, even before Sardana gets migrated. This would be possible thanks to some simple code snippets instantiating the controllers and calling their API methods as this would be done from within the Pool. Have anyone else practice this before? As soon as we have this utility code we will share it. Very optimistic plan is to migrate all our 8 production bemlines before the end of summer 2020.

When migrating plugins, it would be very convenient to make a simultaneous effort to move them from the common repositories of controllers and macros to a dedicated projects as described in SEP16 to facilitate working with branches, etc.

taurel commented 5 years ago

Hi Zibi,

About ESRF machine, we plan to run the MacroServer on Debian 9 hosts within basic conda environment meaning python 3.7.

Cheers

teresanunez commented 5 years ago 
* DESY and ALBA use Debian 9 which comes with Python 3.5 (@teresanunez please confirm)

Yes, @reszelaz, I confirm this

cpascual commented 5 years ago

@reszelaz wrote:

Automatically futurize the develop branch and manually fix eventual problems.

I think that you mean 2to3, not futurize

Futurize would introduce dependencies on the future module in order to provide simultaneous py2+py3 support, which is not what we want for Sardana, AFAIK.

... however, maybe there is some advantage in using futurize and then removing the imports of "future "?

reszelaz commented 5 years ago

I think that you mean 2to3, not futurize

Yes. I thought that futurize had an option to switch between "py3 only" and "py2 and py3". But in this case most probably 2to3 is the tool for us.

reszelaz commented 5 years ago

Thanks @teresanunez and @taurel for the feedback about the Py3 versions!

With all these it seems that we need to support version 3.5.

With respect to the support-2.x branch there is one change. This one will start from the 2.8.0 tag (so from the master branch) - see this similar discussion, instead of the develop branch as it was said earlier. Since 2.8.0 is already tagged we will start this branch tomorrow.

We also plan to run 2to3 as soon as possible (most probably after integrating #1149 into develop). This will be done on a py3 branch pushed to this repository and we will open WIP PR right after that.

On this branch we directly fix the following problems (not fixed by 2to3):

And then we will proceed to fix the failing tests. We will start on the ones affecting the MacroServer. @taurel could you please reply here the features that you use from this documentation or which part of the Macro API do you use. Also if you have a public repo with your macro libraries this may be helpful.

taurel commented 5 years ago

Hi Zibi,

It's difficult for me to answer your questions about our use of the Macro API. As you probably know, for the ESRF machine control system, the MacroServer is used in many different places for many different jobs and by many different groups (ID group, RF group, PS group, Physicists). This means that several different people are writing the macros and I don't really know what's inside these macros. Nevertheless, I think the most sophisticated ones are the macros controlling our Topup system. They are written and managed by our machine physicists. They store these macros in our gitlab at here

I am sorry not to able to give you a more precise answer.

Have a nice week-end

reszelaz commented 5 years ago

Thanks for the macros @taurel ! I quickly reviewed them and it looks like you use:

We will start with testing these features then!