Open dpsanders opened 4 years ago
I like your optimism in using the Open button and seeing what happens!
I guess that this is easy to add in, but hard to do right. With that I mean that there is lots of new UI needed to explain to the user: what they are doing, what the result will be (no two-way conversion), what went wrong (if the import fails) and where they want to save the notebook after importing (overwrite?).
Still, this might be worthwhile to add in, since it makes it easier to get started with Pluto.
I like your optimism in using the Open button and seeing what happens!
Ha. Yes it does seem like a good way to get started with Pluto, but I agree that it seems non-trivial to do correctly.
You could add _pluto
to the filename for example, e.g. hello.jl
-> hello_pluto.jl
, since it's easy enough for the user to change. You don't want to just overwrite their file I guess.
To anyone taking this on: see also #14, maybe this can be combined with Weave.jl and pandoc and Literate.jl and friends
I think a possible way would be through Literate.jl, which has an option to output to Juypter, that means there is already a way to define markdown and Julia cells. By no means I think that a workflow .jl -> pluto should pass through .ipynb, though.
But may be it is even simpler: IMHO it would be sufficient to have syntax to mark cell boundaries, e.g. by # ===
which is recognized by Pluto on the first load and converted into the hash voodoo separators. Then we would have the notion of an "empty", "untouched" "proto-" notebook or however we call it and a notebook with recognized cells and manifest cell order.
This could even go into a Pluto.plutoify() method outside of the browser which takes in a proto notebook and spits out a real one (and complains about errors/ambiguities), very much like Literate itself works. This then could even be used for having a cli tool for this purpose.
Support for writing notebook files by hand is not of this project, and making conversion part of Pluto is requires some carefully designed UI (https://github.com/fonsp/Pluto.jl/issues/132#issuecomment-636748506). But just like Jupyter conversion (https://observablehq.com/@olivier_plas/pluto-jl-jupyter-conversion and https://github.com/vdayanand/Jupyter2Pluto.jl), this can be an external tool, which is very simple to write! Maybe someone could try making it themselves?
If you run Meta.parse(code_string, 1)
, then it parses as much as it can into a single expression, and it returns the index of the next expression:
julia> code="s = 1\nr=2"
"s = 1\nr=2"
julia> Meta.parse(code, 1)
(:(s = 1), 7)
You can repeat this until you have found the start indices of every expression, and you then turn those into Pluto cells.
Hi is there an API to turn something into a pluto cell (md or code) and to write a bunch of cells to a file? I think this is almost all I need...
Yep:
julia> import Pluto
julia> code = ["hello", "world()"];
julia> n = Pluto.Notebook(Pluto.Cell.(code));
julia> path = "hello.jl";
julia> Pluto.save_notebook(n, path);
This gives:
julia> read(path, String) |> Text
### A Pluto.jl notebook ###
# v0.11.4
using Markdown
using InteractiveUtils
# ╔═╡ 844da824-dcb6-11ea-1b3d-95c52106a0d2
hello
# ╔═╡ 844da856-dcb6-11ea-2061-a59a23dd029f
world()
# ╔═╡ Cell order:
# ╠═844da824-dcb6-11ea-1b3d-95c52106a0d2
# ╠═844da856-dcb6-11ea-2061-a59a23dd029f
Ok, this makes it easy to tweak Literate.script(), see my first attempt below. I am not sure if @fredrikekre wants to have a Pluto dependency in Literate.jl. It could also go into Pluto it a Literate dependency is ok, or else into a separate package. To put this into a separate package or into pluto creates some dependencies on internals of Literate, but perhaps this is the best bet.
function pluto(inputfile, outputdir=pwd(); config::Dict=Dict(), kwargs...)
# Create configuration by merging default and userdefined
config = Literate.create_configuration(inputfile; user_config=config, user_kwargs=kwargs)
# normalize paths
inputfile = normpath(inputfile)
isfile(inputfile) || throw(ArgumentError("cannot find inputfile `$(inputfile)`"))
inputfile = realpath(abspath(inputfile))
mkpath(outputdir)
outputdir = realpath(abspath(outputdir))
@info "generating plain script file from `$(Base.contractuser(inputfile))`"
# read content
content = read(inputfile, String)
# run custom pre-processing from user
content = config["preprocess"](content)
# default replacements
content = Literate.replace_default(content, :jl; config=config)
# create the script file
chunks = Literate.parse(content)
code=[]
for chunk in chunks
cell = IOBuffer()
if isa(chunk, Literate.CodeChunk)
for line in chunk.lines
write(cell, line, '\n')
end
write(cell, '\n') # add a newline between each chunk
elseif isa(chunk, Literate.MDChunk)
write(cell,"md\"")
for line in chunk.lines
write(cell, rstrip(line.first * line.second * '\n'))
end
write(cell, "\"\n") # add a newline between each chunk
end
push!(code,String(take!(cell)))
end
# custom post-processing from user
# content = config["postprocess"](String(take!(ioscript)))
# write to file
isdir(outputdir) || error("not a directory: $(outputdir)")
outputfile = joinpath(outputdir, config["name"]::String * ".jl")
if inputfile == outputfile
throw(ArgumentError("outputfile (`$outputfile`) is identical to inputfile (`$inputfile`)"))
end
@info "writing result to `$(Base.contractuser(outputfile))`"
nb = Pluto.Notebook(Pluto.Cell.(code));
Pluto.save_notebook(nb, outputfile);
return outputfile
end
Great! I will try it out soon!
About the dependencies problem - I made PlutoUtils.jl
for scripts like these, having the Literate.jl dependency is fine there. Can you write a PR?
Great! I will try it out soon!
About the dependencies problem - I made
PlutoUtils.jl
for scripts like these, having the Literate.jl dependency is fine there. Can you write a PR?
Yeah I will do this within a day or two...
It should be pretty trivilal to support something like
Literate.notebook(...; flavor = :pluto)
to Literate without having Pluto as a dependency.
For writing a Pluto notebook to disk there is a Pluto API call While it seems to be not too hard to re-create this in Literate.jl without a Pluto dependency, IMHO his would be hard to maintain during the evolution of Pluto. So I think it is better to follow Fons' suggestion to put this into PlutoUtils.
We talked about it here: https://github.com/fonsp/Pluto.jl/issues/311 - the file format will always be backwards compatible, or conversion code is included in Pluto
WIP support for using Literate.jl to create Pluto notebooks from .jl
files: https://github.com/fredrikekre/Literate.jl/pull/120. Please try it out!
... so you have been faster than my PR to PlutoUtils using the Pluto.Notebook constructor... Have no time for testing this immediately. What I would like to see added are some options like fold_markdown_cells
and fold_code_cells
(which I however could add in a later PR).
But see also my 2ct in #311 ...
I tried to open a standard .jl file in Pluto (from the Pluto startup page) and got
Failed to load notebook: MethodError(union, (), 0x0000000000006c87) <a href="/">Go back</a>
It would be great to be able to import a normal
.jl
and have it Plutified. This should e.g. take blocks of code and make them a single cell.
Or perhaps just add a "begin" and "end", then run it on Pluto, and once it is opened in Pluto, delete begin and end and ask Pluto to divide the code into cells.
This won't work, but you can paste the contents into a cell
For the record, I wrote a simple utility for converting plain j.l file to notebook, available at https://github.com/jdadavid/Jl2pluto.jl
I tried to open a standard .jl file in Pluto (from the Pluto startup page) and got
It would be great to be able to import a normal
.jl
and have it Plutified. This should e.g. take blocks of code and make them a single cell.