Enhancement: "Run" markdown files and display the result in the object inspector

[goanpeca]

It would be great to generalize the functionality in the Run menu for "Running/Parsing" source files besides Python scripts.

I am finding myself more often than not, wanting to see how something written in markdown looks rendered, and since the Object Inspector seems to already provide for this functionality, we could make the "Run" behaviour/command more general so that for specific extensions, a behavior could be set.

For this sort of files and rst also, the Run (F5) would also be available, but instead of opening a python console it would render the result in the Object Inspector.

[ccordoba12]

Yes, it would be really, really cool. But for this we would need to add a dependency in mistune and generalize how the Object Inspector works.

[goanpeca]

Is this something that you think would be better as a plugin as well? or is something that could belong to the core?

Is mistune currently being used?

[ccordoba12]

We don't use mistune but IPython will since 3.0.

I don't know how this could be developed as a third-party plugin because it tries to extend a core plugin. So it should belong to the core.

The problem I see with this approach is that we will (probably) need to also extend our Projects support to cover documentation projects (like a Sphinx or mkdocs project only). So things get tricky quite fast :-)

El 23/02/15 a las 10:11, Gonzalo Peña-Castellanos escribió:

Is this something that you think would be better as a plugin as well? or is something that could belong to the core?

Is mistune currently being used?

— Reply to this email directly or view it on GitHub https://github.com/spyder-ide/spyder/issues/2192#issuecomment-75559252.

[goanpeca]

That is precisely my point, maybe Spyder should allow for plugins to extend the core of Spyder as well.

It would be too hard to handle all the possibilities that may arise from here. The basic core Widget (panel...) should handle the most important case. From then on any additional case should be extendable through a plugin.

Lets say that my projects plugin handles Python Packages and thats it, we could have a public API for this specific widget so that it can be extended through a plugin that adds lets say, django, pelican etc. This way the core team would just try to have a simple and stable API to allow for extensions and specific packages can be developed separately on top of this API

In my view of things, plugin need not be a new an unrelated Panel/widget, there should be plugins that enhance the core plugins by extending their functionality, and the Core Widgets/DockWidgets should document a stable Public API to allow for this.

Would not something like this make Spyder truly flexible and powerful?

[ccordoba12]

You're right, that's the best approach in theory. In practice, I don't even know where to start to make that a reality for all plugins (or at least for the most important ones).

That would be a ton of work that we should carefully plan before starting. But sure it'd be interesting to do not only for the Object Inspector but also for the Editor and the Variable Explorer (for starters :-)

El 23/02/15 a las 10:24, Gonzalo Peña-Castellanos escribió:

That is precisely my point, maybe Spyder should allow for plugins to extend the core of Spyder as well.

It would be too hard to handle all the possibilities that may arise from here. The basic core Widget (panel...) should handle the most important case. From then on any additional case should be extendable through a plugin. Lets say that my projects plugin handles Python Packages and thats it, we could have a public API for this specific widget so that it can be extended through a plugin that adds lets say, django, pelican etc. This way the core team would just try to have a simple and stable API to allow for extensions and specific packages can be developed separately on top of this API

In my view of things, plugin need not be a new an unrelated Panel/widget, there should be plugins that enhance the core plugins by extending their functionality.

— Reply to this email directly or view it on GitHub https://github.com/spyder-ide/spyder/issues/2192#issuecomment-75563914.

[goanpeca]

"Nobody said it was easyyyyyyy...." haha

Not saying it has to be done at once and for all plugins, so we could use this as an opportunity to add the behavior (an extendable simple API) to the object inspector and see how it goes or could go for other plugins.

I already have in mind how to split the editor widget. The editor widget is hard wired for python so I want to split it into a base editor and a specific python on top of it. If this base widget is general enough it could handle a large number of use cases. From looking at the PyQode source code I have now a much clearer view on how to achieve this.

For instance.... spoiler ...crazy idea here....

With conda we will eventually be able to install R and R packages, we could have a basic Console Widget and a Basic Editor widget that is extended (with a plugin) to add support for the R console and R completion etc etc....

[ccordoba12]

1. Starting with the Object Inspector for a public APi is a great idea because it's perhaps the easiest plugin.
2. It'd be really cool to adapt our Editor to work with R, julia, octave, etc.

[goanpeca]

Ok, I haven't looked at the object inspector, so far, so I will take a look at it to get familiar with how it works and think of how it could be made extendable.

[goanpeca]

Ok I have been looking at the run thinguie in the editor plugin (never had before...) wow... that is a big monster :)!

I have some ideas but we need to figure out how these plugins (the installable ones I mean) should be incorporated. Should each dockwidget plugin has some machinery to look for plugins that extend its functionality? or should Spyder as a final step of setup go through all installed plugins and modify the core dockwidget plugins accordingly (depending if the plugin is enable or not)?

--- side note: the name plugins brings confusion, "Editor Plugin" is a core dockwidget, but we have also plugins to be installed eventually, but these words do not relate to each other ---

The first thing I think of changing with how it is currently set is, get rid of the is_python method in the sourcecode editor and replace it for a more global way (language agnostic) to get the filetype or the language and from that language a dictionary could hold all relevant actions for the language (for instance in the run menu). When a plugin is installed it could add actions to this dictionary (part of the public API) so that when the run menu is rendered it would adjust the actions (instead of graying them out), so if a markdown fie is open, only a Parse action (F5) would be seen instead of 7 grayed out actions. Instead if having a variable holding the RUN menu, we could have the variable hold a dictionary with base actions, and a set of actions defined for each language (that is added via a plugin)

A similar principle could hold for all the other menus, except in menus that affect ALL types of files, the actions to add should be placed in a specific section only... as to not get an additional action below the Quit button in the File menu, which would make no sense!

To do this we should define some global constants to refer to the languages.... something like:

spyder.constants.languages.PYTHON
spyder.constants.languages.MARKDOWN
spyder.constants.languages.UNDEFINED

The sourcecodeeditor should have then a method like... get_language that would return what at the init looked if it is .md or .markdown... and return spyder.constants.languages.MARKDOWN or spyder.constants.languages.UNDEFINED, in which case no special actions should be added (much like what happens now with any language besides python)

With all in place, then we could look at the object inspector, which actually would not need much change as the way I see it would just accept some input with a new method render_this_html(self, html) tied to the added Run/Parse action in the run menu. Also this plugin would have mistune as a dependency, and maybe a version of the Editor and Object Inspector Public API. If we have a version for the API of each dockwidget it would be easier to control changes per dockwidget plugin and also have checks to ensure that an installable plugin can work. Also we can grow the API for each dockwidget gradually, instead of tying ALL the api to a specific version.

@ccordoba12 please let me know what you think, or any ideas to add on this.

[ccordoba12]

I don't like too much your proposal. Why not go simple at first and just define a new run_markdown function, that would be called by F5 when the file type is markdown?

I mean, what else needs to be added in the Run menu that justifies the type of abstraction you're suggesting?

Also pinging @blink1073 for his opinion because he has worked a lot with codeeditor too.

[goanpeca]

My proposal is aiming to have a more generic way of handling different language filetypes.

Adding a run_markdown function in the object inspector is just what I propose, but I was suggesting trying to generalize the use plugins to extend functionality (in the editor in this case), otherwise we will keep adding special cases. Right now doing the "simple at first" approach is fine to get the job done (rendering markdown files) but I am looking further than this specific issue.

Also my proposal was to add this as an external plugin, to "ease the burden" on the core of Spyder (even if it is a simple thing), plus it can serve as an example for other plugins to be written.

Adding this sort of genericity will be needed for the sourcecode editor in order to implement language specific helper functions. I think in the medium and long term it is not sustainable to bundle everything inside the plugins with special cases is_makdown, is_python, is_python_like, is_cython, is_r etc etc etc...

Use cases:

• C: Don't know, lets say you open a C file for a function you want to optimize... the run here would maybe add options to point to a compiler.
• Cython: Lets say we are running cython files, the Run menu could include something extra
• R: the run would open an specific R console that would be another pluign lets say
• HTML: the file could be rendered in a dedicated QWebkit? like what will be used for the notebooks editing?

[ccordoba12]

About the use cases you propose:

• C: It could be interesting, but more interesting would be to run Fortran files
• Cython: Definitely, several people have been asking for it.
• R: With the new IPython kernel spec, it's possible to open an R console inside Spyder with an R kernel. Using it we can also create Ruby, Perl, FSharp, etc, etc, consoles.
• HTML: Mmmh, that's what browsers are for ;-)

But as I see it, we don't need to go too deep refactoring code. The Run menu doesn't need to change, cells can continue working as expected. The real problem we need to solve before refactoring anything is define how to extend a core plugin by a third-party one. I think your proposal doesn't address that, and it's not clear to me how to do that either.

[goanpeca]

Fair enough I need to polish the details, but before anything, we need to address how Spyder will handle normal Plugins, how it will autodetect them and how they will be installed/activated/disactivated.

Do you have an idea on that already?

[blink1073]

I think we just need to provide a way to extend plugins by name. For example, we allow other plugins to provide a "run" target tied to the run plugin. We collect the extensions at startup, then if a file is opened, we check our list of registered extensions. If one is available, the Run option becomes available, and the registered function is called when the user presses F5.

[blink1073]

I think we could allow users to keep plugins and extensions in their ~/.spyder directory, similar to how IPython does it.

[goanpeca]

I think it would be better to make plugins pip/conda installable, this way it would be a matter of clicks and not download unzip store in a hidden folder....

[blink1073]

The nice thing about the IPython way is that they are all in one place and not cluttering up site-packages.

[goanpeca]

How is that benefiting the end user?

[blink1073]

Namespace and separation of concerns. IPython extensions/kernels are still pip installable, they just get sent to the .ipython folder.

[goanpeca]

Ok, not sure about the details of where a package/plugin will get stored. I think that where they get stored is besides the point, as long as they can be installed via pip (or conda).

Maybe for us (developers) it is nicer to keep them in one place, that I agree, but for the end user will be irrelevant, so if the can be installed in .spyder/plugins great!

@blink1073 , what do you think of reorganizing the editor so that there is a base editor and then language specific enhancements? (also installable as plugins)

[blink1073]

My opinion is that Spyder should be language agnostic as much as possible. Even folks that primarily use Python also use C / Octave / Markdown / etc. I agree your approach. However, I think we should concentrate on the Scientific Python parts of that equation while allowing for outside contributions.

[goanpeca]

Good :)

Sorry what do you mean by this "concentrate on the Scientific Python parts of that equation" ?

[blink1073]

Namely Python and Cython :)

[goanpeca]

ahhhhhhhh sure :)

[ufechner7]

Dont't forget Julia.

[Nodd]

For Markdwn precisely, instead of having to run the file by F5 the preview could be shown in "realtime" (either on each file save or after series of keystrokes) in a specific dockwidget. If it's only one file the execution should be really fast. Still for other languages a custom Run option would be great !

@ufechner7 I would love to use julia, but it uses fftw for its fft functions which is GPL :(

[goanpeca]

In a specific dockwidget... it feels like too many dockwidgets already no?, couldnt it be just parsed and showed in the object explorer in real time then?

Julia: But as a plugin it would be still ok, no?

[Nodd]

Editing markdown is a specific task so I wouldn't care having a specific dockwidget, especially if it's an external plugin. Reusing the object explorer is fine but it would have to get another name !

[goanpeca]

Well.. I think @ccordoba12 wanted to make it help center or something

[Nodd]

About julia, I was talking about using it at work. This seem a very good language for scientific development but this license is a problem for me. Intregration in spyder is fine as long as we don't ship the library itself.

[dougthor42]

I would love to have this for reStructuredText in addition to Markdown.

What I currently do it write my reST in the docstring of a dummy function and then inspect it with Ctrl+I. After I'm done formatting, I'll copy the reST into my readme.rst file (or wherever) and be done.

The other way I'll write reST files is with restview - it opens up a localhost webserver and which renders the document, updating on manual refresh (and on document save, I think). Pretty nifty, and there's a similar things for Markdown mdview though I've never tried it.

Adding the ability to view formatted .md or .rst files would be very nice.

On a related note, I'd also like to be able to view the rendered module docstring, but I'll open up a separate issue for that.

edit: fixed capitalization of 'reStrucutredText' and 'reST'

[Nodd]

For information I use ReText to edit Rest or markdown files. It can display a rendering in realtime. The only limitation I found for my use is that it doesn"t understand the sphinx additions to rst.

[goanpeca]

I think we will add this as a native thing in spyder, so hopefully @Nodd wont need to go to ReText ;-)

[Nodd]

I was just providing a possible alternative in the meantime, that's all ! I rarely edit rst files actually, these days I tend to create them automatically to show computation results (python powa!)

[ccordoba12]

This is going to be done by spyder-reports, so closing.