Closed mzimmerm closed 6 months ago
Gilad,
This commit/pull request contains changes discussed on Google Groups in https://groups.google.com/g/newspeaklanguage/c/zzzl2xorMOE towards the end of the thread.
The core of the change was to allow the PWA version for newspeaklanguage.org to be kept in git, but git-ignore the PWA versions for localhost or other deployments by default.
As discussed there, the change needed to create a new deploy script, during which the PWA version increase and change in a git directory is performed.
As you may see from the text below and from the pull request, my changes went further than discussed, and I appologize for not communicating this fully.
Although everything seems to work well, I would understand if you did not like it for various reasons.
Anyway, my goals were to:
The very brief summary of the changes are:
How it changes life of someone who used to use the system before the change:
There is an oversized change in README.md describing this change, and how to build the system with the changes in this commit. Section 4.1 and 4.2 of the README are the core.
I am sure you will have follow ups, we can do either here or on Google groups.
Hope this will work for you, but I am here to make any changes,
Milan
In my last comment https://github.com/newspeaklanguage/newspeak/pull/125#issuecomment-1933542371 in section
"How it changes the life of someone who used to use the system before the change:"
I forgot to add an important note. This commit adds a file
tool/newspeak_env.sh
This file is sourced in all scripts. It defines the following variables and their 'default' directory names:
export EMSDK=~/software/emsdk # Emscripten directory
export NEWSPEAK=~/software/nswasm/newspeak # Newspeak directory
export PRIMORDIALSOUP=~/software/nswasm/primordialsoup # Primordialsoup directory
This file needs to be modified if the user prefers different directories. This must be done before running the 'tool/build.sh' or the 'deploy*' scripts.
Gilad, there is no rush at all. Thanks for the follow-up.
Hi Gilad,
Thank you for reviewing everything in detail.
I am thinking about the following approach to implement your suggestions:
I should be able to finish my full follow-up for items 2 and 3 tonight, but more likely tomorrow.
I hope this will work
Thank you, Milan
On Mon, Feb 12, 2024 at 6:17 PM Gilad Bracha @.***> wrote:
@.**** commented on this pull request.
Hi Milan. I left some comments. Most are just about minor typos/English, but a few are slightly more meaningful. In some cases, it's clear we should discuss a bit. If you could please address these and we can move forward soon. Thanks for a very thorough job!
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486778506 :
-Set up a directory for your work, say, nswasm +A Newspeak application is a Newspeak class which adheres to a few specific API naming conventions. This class can be saved to a
.ns
source file, then 'build' (compiled) to a binary.vfuel
, which is deployed and served.build -> built
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486779723 :
-and then clone this repo under it. +1. From the locally served or online Newspeak IDE by clicking the [deploy] link beside the application class. This process can be slow, currently not working (todo is it not?), and is not described here.
the link shoukd probably be to the ide itself, i.e., https://newspeaklanguage.org/webIDE/
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486782889 :
-Follow the instructions at +TL;DR: If you want to build the Newspeak IDE or any Newspeak application from scratch, follow the intallation steps in the section 3.1 and subsection 3.1.1 (one-time only). Then follow the steps in section 4.1 (for Newspeak IDE running as PWA) or 4.2 (for application running as a website). The steps in section 4.1 or 4.2 are repeated after after any change in the application.
the section 3.1 > section 3.1
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486788356 :
-In the top level of the newspeak repository, you should find the script -build.sh. It assumes Emscripten is in ~/software/emsdk/; you will -want to adjust that to reflect where you have installed emscripten. +On a high level, to build Newspeak locally requires some software to be downloaded or installed: Newspeak, Primordialsoup and it's dependecies and targets: Emscripten, Scons, g++-multilib. Details of the installing the required software are below.
to build -> building the installing ->how to install
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486792347 :
-You can access it by running a web server, using the serve.sh script: +- Adjust the above directory names in this document +- After cloning Newspeak in the steps below, edit the file
newspeak/tool/newspeak_env.sh
and adjust the directory names to your situation.I'd rephrase this (from line 52 on) to say something like: ... the default directories are determined by environment variables set in newspeak/tool/newspeak_env.sh. These variables and there default values are . We use these defaults throughout the rest of this document.
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486793525 :
-http://localhost:8080/primordialsoup.html?snapshot=HopscotchWebIDE.vfuel +1. Start with setting up a directory for Newspeak and Primordialsoup,
~/software/nswasm
and then clone this repo under it. +2. In the same directory, cloneTo avoid ambiguity, I'd say:
- In ~/software/nswasm clone
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486795256 :
+Important: If you are using 'non default' directories for your installation, now edit the
newspeak/tool/newspeak_env.sh
, change the exported variables there, and save the file. The default uneditednewspeak/tool/newspeak_env.sh
is: + +```sh +export EMSDK=~/software/emsdk # Emscripten directory +export NEWSPEAK=~/software/nswasm/newspeak # Newspeak directory +export PRIMORDIALSOUP=~/software/nswasm/primordialsoup # Primordialsoup directoryIn my telling, you'll have specified all this mapping earlier.
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486798227 :
- Clone the Emscripten SDK (
emsdk
) using these stepsSet the Newspeak environment
- cd ~/software/nswasm/newspeak # Adjust to your directory path
- . ./tool/newspeak_env.sh
What's the extra . here? Should just be ./tool/newspeak_env.sh (I assume it's executable).
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486800108 :
+
List the emsdk installed
- ./emsdk list
- Note: '2.0.0' is the default supported version of
emsdk
. To use the latest Emscripten SDK, you can replace '2.0.0' with 'latest' in the script above. BUT, if you do that, you also have to edit the scons build file namedSConstruct
in the$PRIMORDIALSOUP
directory, and replace the following line- '-s', 'EXPORTED_FUNCTIONS=["_load_snapshot", "_handle_message", "_handle_signal"]',
- with
- '-s', 'EXPORTED_FUNCTIONS=["_load_snapshot", "_handle_message", "_handle_signal", "_free", "_malloc"]',
- '-s', 'EXPORTED_RUNTIME_METHODS=["stringToUTF8"]',
- to be clear, the first (default) works with
emsdk
version '2.0.0', the second with 'latest'.Maybe we should just set things up to work with latest? What do you think?
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486801831 :
- cd ~/software/nswasm/primordialsoup # Adjust to your directory path
- . ../newspeak/tool/newspeak_env.sh # Set EMSDK var for next line
- . $EMSDK/emsdk_env.sh # Setup emsdk's environment
- ./build os=emscripten arch=wasm # Build wasm using emscripten
- There should be no errors in the last step, and you should see directories in the
out
directory underprimordialsoup
, notably there should be filesprimordialsoup/out/ReleaseEmscriptenWASM/{primordialsoup.html,primordialsoup.js,primordialsoup.wasm}
andprimordialsoup/out/snapshots/WebCompiler.vfuel
which are used in the Newspeak build. If there are errors, you may be missing someprimordialsoup
dependencies described in the section above.+Note: For original instructions of Primordialsoup dependencies that include Fuchsia as a target, follow the instructions at https://github.com/newspeaklanguage/primordialsoup/blob/master/docs/building.md.
+## 3.2 Building and deploying Newspeak applications: lifecycle
+The above section Building Newspeak: installing required software including its subsection Building Newspeak: installing Primordialsoup dependencies are only need to be performed one-time during installation.
are only -> only
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486803836 :
+
- There should be no errors in the last step, and you should see directories in the
out
directory underprimordialsoup
, notably there should be filesprimordialsoup/out/ReleaseEmscriptenWASM/{primordialsoup.html,primordialsoup.js,primordialsoup.wasm}
andprimordialsoup/out/snapshots/WebCompiler.vfuel
which are used in the Newspeak build. If there are errors, you may be missing someprimordialsoup
dependencies described in the section above.+Note: For original instructions of Primordialsoup dependencies that include Fuchsia as a target, follow the instructions at https://github.com/newspeaklanguage/primordialsoup/blob/master/docs/building.md.
+## 3.2 Building and deploying Newspeak applications: lifecycle
+The above section Building Newspeak: installing required software including its subsection Building Newspeak: installing Primordialsoup dependencies are only need to be performed one-time during installation.
+This section and sections below describe the repeated 'build', 'deploy', 'serve' lifecycle of Newspeak applications.
+You need to run the 'build' and 'deploy' steps in one of the sections below after you made a change to a Newspeak source, and saved it to a
.ns
file. The changed file(s) may be one of the Newspeak core classes, or a class that is a part of your application you are building in Newspeak. You need to run a 'build' to regenerate the application's.vfuel
file, and run one of the 'deploy' scripts to copy the results to the 'deployment directory'. You also need to start the HTTP server in the 'deployment directory'.+Important: All scripts to build and deploy a Newspeak application are in the
newspeak/tool
directory, and must be executed from thenewspeak/tool
directory. The Newspeak IDE is only one of possible Newspeak applications.The Newspeak IDE is only one of possible Newspeak applications -> The Newspeak IDE is only one possible Newspeak application
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486806908 :
+The above section Building Newspeak: installing required software including its subsection Building Newspeak: installing Primordialsoup dependencies are only need to be performed one-time during installation. + +This section and sections below describe the repeated 'build', 'deploy', 'serve' lifecycle of Newspeak applications. + +You need to run the 'build' and 'deploy' steps in one of the sections below after you made a change to a Newspeak source, and saved it to a
.ns
file. The changed file(s) may be one of the Newspeak core classes, or a class that is a part of your application you are building in Newspeak. You need to run a 'build' to regenerate the application's.vfuel
file, and run one of the 'deploy' scripts to copy the results to the 'deployment directory'. You also need to start the HTTP server in the 'deployment directory'. + +Important: All scripts to build and deploy a Newspeak application are in thenewspeak/tool
directory, and must be executed from thenewspeak/tool
directory. The Newspeak IDE is only one of possible Newspeak applications. + +The term build refers to the process which creates the.vfuel
binary files from the application source.ns
files. The scriptnewspeak/tool/build.sh
is the build script. + +The term deploy refers to the process of copying the Newspeak application.vfuel
file and other files and resources (.html
,.js
,.png
) to the 'deploy directory' where an HTTP server is or will be running. The 'deploy' step also copies scripts that can start an HTTP server to the 'deploy directory'. Newspeak applications can be deployed in two ways: as a Progressive Web App (PWA) or as a regular website. The 'PWA deploy' is performed by the scriptnewspeak/tool/deploy-webide-vfuel-as-pwa.sh
, the 'website deploy' is performed by the scriptdeploy-all-vfuels-as-website.sh
+ +The term serve refers to running the HTTP server in a directory where the Newspeak application was 'deploy'ed. + +Note: If you want to deploy other application's.vfuel
as a PWA, you can use the directory structure under webIDE, rename it, make changes toindex.html
,manifest.json
,sw.js
,primordialsoup-setup.js
, also remove files not applicable to your application, and copy your application's.vfuel
in the location ofHopscotchWebIDE.vfuel
. +use -> clone
copy your application's .vfuel in the location of HopscotchWebIDE.vfuel -> replace HopscotchWebIDE.vfuel with your application's .vfuel
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486810499 :
+Note: For original instructions of Primordialsoup dependencies that include Fuchsia as a target, follow the instructions at https://github.com/newspeaklanguage/primordialsoup/blob/master/docs/building.md. + +## 3.2 Building and deploying Newspeak applications: lifecycle + +The above section Building Newspeak: installing required software including its subsection Building Newspeak: installing Primordialsoup dependencies are only need to be performed one-time during installation. + +This section and sections below describe the repeated 'build', 'deploy', 'serve' lifecycle of Newspeak applications. + +You need to run the 'build' and 'deploy' steps in one of the sections below after you made a change to a Newspeak source, and saved it to a
.ns
file. The changed file(s) may be one of the Newspeak core classes, or a class that is a part of your application you are building in Newspeak. You need to run a 'build' to regenerate the application's.vfuel
file, and run one of the 'deploy' scripts to copy the results to the 'deployment directory'. You also need to start the HTTP server in the 'deployment directory'. + +Important: All scripts to build and deploy a Newspeak application are in thenewspeak/tool
directory, and must be executed from thenewspeak/tool
directory. The Newspeak IDE is only one of possible Newspeak applications. + +The term build refers to the process which creates the.vfuel
binary files from the application source.ns
files. The scriptnewspeak/tool/build.sh
is the build script. + +The term deploy refers to the process of copying the Newspeak application.vfuel
file and other files and resources (.html
,.js
,.png
) to the 'deploy directory' where an HTTP server is or will be running. The 'deploy' step also copies scripts that can start an HTTP server to the 'deploy directory'. Newspeak applications can be deployed in two ways: as a Progressive Web App (PWA) or as a regular website. The 'PWA deploy' is performed by the scriptnewspeak/tool/deploy-webide-vfuel-as-pwa.sh
, the 'website deploy' is performed by the scriptdeploy-all-vfuels-as-website.sh
+What I found confusing is that the PWA option is specific to the IDE while the other option is not. I understand why (setting up a PWA requires some more app-specific work) but when you say that Newspeak apps can be deployed in two ways (which is true) and then introduce 2 scripts, one is liable to think that the scripts are both universal and be confused by the naming. What I suggest:
(1) reverse the order: Newspeak applications can be deployed in two ways: as a regular website or as a Progressive Web App (PWA). (2) Then introduce the general purpose script - the 'website deploy' is performed by the script deploy-all-vfuels-as-website.sh (3) Next, explain that only the IDE is currently set up to be deployed as a PWA, and it can be deployed using newspeak/tool/deploy-webide-vfuel-as-pwa.sh.
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486817562 :
+# and places them to the 'out' directory. +# Note: The listed '.vfuel' files include the 'HopscotchWebIDE.vfuel'. + +cd ~/software/nswasm/newspeak # Adjust to your directory path +cd tool +./build.sh + +# Deploy Step: +# +# Deploys the 'HopscotchWebIDE.vfuel' as a Progressive Web App (PWA) (as opposed to a website). +# Copies all built files to the Newspeak HTTPs server directory +# specified in the first argument, and updates the PWA Service Worker version file +# specified in the second argument. The updated version is used as the PWA cache version. +# By 'All built files' we mean the 'HopscotchWebIDE.vfuel' and miscelaneous js and html files +# supporting the PWA, as well as the script 'serve-newspeak.sh', which starts the HTTP server. +#
miscelaneous -> miscellaneous
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486818959 :
+# +# Usage Examples: +# +# 1. Deploy files for server running from +# the '~/servers/newspeak-http-server' directory, serving as 'localhost' +# - cd tool +# - ./deploy-webide-vfuel-as-pwa.sh ~/servers/newspeak-http-server localhost +# +# 2. Deploy files for server running from +# the '~/servers/newspeak-http-server' directory, serving as 'newspeaklanguage.org' +# - cd tool +# - ./deploy-webide-vfuel-as-pwa.sh ~/servers/newspeak-http-server newspeaklanguage.org +# +# 'newspeaklanguage.org' is a special case in the sense that git does NOT ignore +# it's version file, the '$NEWSPEAK/pwa-deployed-versions/newspeaklanguage.org.version'. +# This ensures a global uniqueness of the PWA version served from 'https://newspeaklanguage.org'.
a global -> the global
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486821035 :
+# +# 1. Deploy files for server running from +# the '~/servers/newspeak-http-server' directory, serving as 'localhost' +# - cd tool +# - ./deploy-webide-vfuel-as-pwa.sh ~/servers/newspeak-http-server localhost +# +# 2. Deploy files for server running from +# the '~/servers/newspeak-http-server' directory, serving as 'newspeaklanguage.org' +# - cd tool +# - ./deploy-webide-vfuel-as-pwa.sh ~/servers/newspeak-http-server newspeaklanguage.org +# +# 'newspeaklanguage.org' is a special case in the sense that git does NOT ignore +# it's version file, the '$NEWSPEAK/pwa-deployed-versions/newspeaklanguage.org.version'. +# This ensures a global uniqueness of the PWA version served from 'https://newspeaklanguage.org'. +# +# 3. Deploy files for server running from
from where? out?
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486821816 :
+# 1. Deploy files for server running from +# the '~/servers/newspeak-http-server' directory, serving as 'localhost' +# - cd tool +# - ./deploy-webide-vfuel-as-pwa.sh ~/servers/newspeak-http-server localhost +# +# 2. Deploy files for server running from +# the '~/servers/newspeak-http-server' directory, serving as 'newspeaklanguage.org' +# - cd tool +# - ./deploy-webide-vfuel-as-pwa.sh ~/servers/newspeak-http-server newspeaklanguage.org +# +# 'newspeaklanguage.org' is a special case in the sense that git does NOT ignore +# it's version file, the '$NEWSPEAK/pwa-deployed-versions/newspeaklanguage.org.version'. +# This ensures a global uniqueness of the PWA version served from 'https://newspeaklanguage.org'. +# +# 3. Deploy files for server running from +# This mimicks (I think) how the server ran before this change.
We'll have to verify this.
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486827677 :
+# - cd tool +# - ./deploy-webide-vfuel-as-pwa.sh ~/servers/newspeak-http-server newspeaklanguage.org +# +# 'newspeaklanguage.org' is a special case in the sense that git does NOT ignore +# it's version file, the '$NEWSPEAK/pwa-deployed-versions/newspeaklanguage.org.version'. +# This ensures a global uniqueness of the PWA version served from 'https://newspeaklanguage.org'. +# +# 3. Deploy files for server running from +# This mimicks (I think) how the server ran before this change. +# The advantage of serving from the '../out' directory is that +# You ONLY need to run the 'deploy' script once, after, just keep running the 'build' script. +# - cd tool +# - ./deploy-webide-vfuel-as-pwa.sh ../out localhost + +httpServerRunPath=~/servers/newspeak-http-server # Adjust to where you want HTTP server running +
Here the narrative structure seems to break down. Why and when are we setting httpServerRunPath?
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486828782 :
+Running the above commands would build, deploy, and serve the
HopscotchWebIDE.vfuel
as PWA.as PWA -> as a PWA
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486838199 :
+# Build Step: +# Should be run at least once, and after any change to a Newspeak source file. +# Builds all '.vfuel' files listed in the 'build.sh' +# and places them to the 'out' directory. + +cd ~/software/nswasm/newspeak # Adjust to your directory path +cd tool +./build.sh + +# Deploy Step: +# +# Deploys all '.vfuel' files listed in 'build.sh' as a website (as opposed to PWA). +# Copies all built files to the Newspeak HTTPs server directory +# specified in the first argument. +# By 'All built files' we mean all '.vfuel' files built by 'build.sh', +# and miscelaneous js and html files supporting the website.
miscelaneous -> miscellaneous
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486840580 :
+# 1. Deploy files for server running from +# the '~/servers/newspeak-http-server' directory +# - cd tool +# - ./deploy-all-vfuels-as-website.sh ~/servers/newspeak-http-server +# 2. Deploy files for server running from +# the '../out' directory. +# This mimicks (I think) how the server ran before this change +# The advantage of serving from the '../out' directory is that +# You ONLY need to run this 'deploy' script once, after, just keep running the 'build' script. +# - cd tool +# - ./deploy-all-vfuels-as-website.sh ../out + +httpServerRunPath=~/servers/newspeak-http-server # Adjust to where you want HTTP server running + +./deploy-all-vfuels-as-website.sh $httpServerRunPath +
Not clear about the role of these two lines.
In pwa-deployed-versions/newspeaklanguage.org.version https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486843122 :
@@ -0,0 +1 @@ +30
We'll need to set this number correctly when we merge this in.
In tool/build.sh https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486855578 :
+cp ${PRIMORDIALSOUP}/newspeak/.ns out +# 7. Merge in Newspeak IDE dependencies +cp ./.ns ./.png out +# cp -R CodeMirror out # CodeMirror is copied during deploy + +cd ${NEWSPEAK}/out || exit 1 + +# ${PRIMORDIALSOUP}/out/ReleaseX64/primordialsoup \ +# ${PRIMORDIALSOUP}/primordialsoup/out/snapshots/WebCompiler.vfuel \ +# .ns \ +# .png \ +# not in 'out': .txt \ +# not needed, included in html both index.html for WebIDE and primordialsoup.html: CodeMirror/lib/codemirror.js \ +# not needed, included in html both index.html for WebIDE and primordialsoup.html: CodeMirror/addon/display/autorefresh.js \ +# RuntimeForHopscotchForHTML HopscotchWebIDE HopscotchWebIDE.vfuel \ +# RuntimeForElectron HopscotchWebIDE HopscotchElectronIDE.vfuel \
While I still find Electron occasionally useful, we aren't actually distributing an executable version anymore. We should think about removing it from here.
In tool/build.sh https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486856833 :
+# cp -R CodeMirror out # CodeMirror is copied during deploy + +cd ${NEWSPEAK}/out || exit 1 + +# ${PRIMORDIALSOUP}/out/ReleaseX64/primordialsoup \ +# ${PRIMORDIALSOUP}/primordialsoup/out/snapshots/WebCompiler.vfuel \ +# .ns \ +# .png \ +# not in 'out': *.txt \ +# not needed, included in html both index.html for WebIDE and primordialsoup.html: CodeMirror/lib/codemirror.js \ +# not needed, included in html both index.html for WebIDE and primordialsoup.html: CodeMirror/addon/display/autorefresh.js \ +# RuntimeForHopscotchForHTML HopscotchWebIDE HopscotchWebIDE.vfuel \ +# RuntimeForElectron HopscotchWebIDE HopscotchElectronIDE.vfuel \ +# RuntimeForHopscotchForHTML Ampleforth Ampleforth.vfuel \ +# RuntimeForHopscotchForHTML AmpleforthViewer AmpleforthViewer.vfuel \ +# not in git: RuntimeForHopscotchForHTML Live22Submission Live22Submission.vfuel \
Yes. Let's remove all these. I will have my own local script to do this as needed.
In tool/build.sh https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486860614 :
+ +# ${PRIMORDIALSOUP}/out/ReleaseX64/primordialsoup \ +# ${PRIMORDIALSOUP}/primordialsoup/out/snapshots/WebCompiler.vfuel \ +# .ns \ +# .png \ +# not in 'out': *.txt \ +# not needed, included in html both index.html for WebIDE and primordialsoup.html: CodeMirror/lib/codemirror.js \ +# not needed, included in html both index.html for WebIDE and primordialsoup.html: CodeMirror/addon/display/autorefresh.js \ +# RuntimeForHopscotchForHTML HopscotchWebIDE HopscotchWebIDE.vfuel \ +# RuntimeForElectron HopscotchWebIDE HopscotchElectronIDE.vfuel \ +# RuntimeForHopscotchForHTML Ampleforth Ampleforth.vfuel \ +# RuntimeForHopscotchForHTML AmpleforthViewer AmpleforthViewer.vfuel \ +# not in git: RuntimeForHopscotchForHTML Live22Submission Live22Submission.vfuel \ +# not in git: RuntimeForHopscotchForHTML Smalltalks22Tutorial Smalltalks22.vfuel \ +# not in git: RuntimeForHopscotchForHTML Live22Presentation Live22Presentation.vfuel \ +# RuntimeForCroquet CounterApp CroquetCounterApp.vfuel \
The Croquet version (and most of the others below) also need not be built by default. I'm not sure where to keep my customized scripts for these. Maybe still in the Newspeak repo?
In tool/build.sh https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486861700 :
+# RuntimeForHopscotchForHTML TodoMVCApp TodoMVCApp.vfuel \ +# RuntimeForCroquet TodoMVCApp CroquetTodoMVCApp.vfuel \ +# RuntimeForHopscotchForHTML TwoViewEditorApp TwoViewEditorApp.vfuel \ +# RuntimeForHopscotchForHTML TelescreenApp Telescreen.vfuel \ +# RuntimeForHopscotchForHTML ObjectPresenterDemo ObjectPresenterDemo.vfuel \ +# RuntimeForHopscotchForHTML BankAccountExemplarDemo BankAccountExemplarDemo.vfuel \ +# RuntimeForHopscotchForHTML HopscotchFontDemo HopscotchFontDemo.vfuel \ +# RuntimeForHopscotchForHTML HopscotchGestureDemo HopscotchGestureDemo.vfuel \ +# RuntimeForHopscotchForHTML HopscotchDemo HopscotchDemo.vfuel \ +# RuntimeForHopscotchForHTML Particles Particles.vfuel + +# +# 8. Create vfuel files (ex: HopscotchWebIDE.vfuel) +# from ns files (ex: HopscotchWebIDE.ns) and +# additional resources (.ns, .png). +#
Ok, this section seems to be duplicated.
In tool/deploy-all-vfuels-as-website.sh https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486895703 :
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + + +# Description: +# +# Deploys all '.vfuel' files listed in 'build.sh' as a website (as opposed to PWA). +# Copies all built files to the Newspeak HTTPs server directory +# specified in the first argument. +# By 'All built files' we mean all '.vfuel' files built by 'build.sh', +# and miscelaneous js and html files supporting the website.
miscelaneous -> miscellaneous
In tool/deploy-all-vfuels-as-website.sh https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486896885 :
+ +# Arguments: +# +# 1. http-server-dir +# Note: there is no need for the second argument as in the PWA script - +# we do not need a PWA version number. +# See function 'args_parse_for_deploy_as_website' for details + +# Usage Examples: +# 1. Deploy files for server running from +# the '~/servers/newspeak-http-server' directory +# - cd tool +# - ./deploy-all-vfuels-as-website.sh ~/servers/newspeak-http-server +# 2. Deploy files for server running from +# the '../out' directory. +# This mimicks (I think) how the server ran before this change
Once we're sure, we should get rid of "I think" everywhere
In tool/newspeak_env.sh https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486903504 :
@@ -0,0 +1,16 @@ +#!/bin/sh + +## +## Description: +## This file is sourced in any script running Newspeak build.
Yes, but indirectly, via newsepak_util.sh; might be worth saying that.
In tool/newspeak_util.sh https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486908852 :
- rsync -r "$NEWSPEAK/platforms/webIDE" "$httpServerDir"
Copy results of newspeak/tool/build.sh to the server directory
1. Of all .vfuels, ONLY the HopscotchWebIDE.vfuel is copied, so this runs ONLY webIDE
- rsync "$NEWSPEAK/out/HopscotchWebIDE.vfuel" "$httpLibDir/"
2. Copy primordialsoup.wasm and primordialsoup.js
- rsync "$NEWSPEAK/out/primordialsoup.wasm" "$httpLibDir/"
- rsync "$NEWSPEAK/out/primordialsoup.js" "$httpLibDir/"
Increase the pwaVersion managed in pwaVersionPath
and copy the 'sw.js' with the increased version to the webIDE
- copy_sw_to_http_dir_increasing_pwa_version +}
+# +# Copies all necessary file to run the Newspeak HTTP server serving
file -> files
In README.md https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1486781293 :
-and then clone this repo under it. +1. From the locally served or online Newspeak IDE by clicking the [deploy] link beside the application class. This process can be slow, currently not working (todo is it not?), and is not described here.
It does work, though it is a bit slow.
— Reply to this email directly, view it on GitHub https://github.com/newspeaklanguage/newspeak/pull/125#pullrequestreview-1876317421, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALGQ7AKUEHXDWYH5KMIHO3YTKPI7AVCNFSM6AAAAABC7HD4F6VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTQNZWGMYTONBSGE . You are receiving this because you authored the thread.Message ID: @.***>
Sounds good. Thanks.
Gilad,
I finished making changes from the review-suggested comments in this https://github.com/newspeaklanguage/newspeak/pull/125. (Well, there were a few review items where I am unsure if you prefer to remove comments now or after further review. Such items are either text or comments in code, we can address them in the next step.)
Many of the changes were trivial spelling and simple grammar errors, I marked those as 'done'.
To reduce your load going over all changes again, here are the first line numbers of the non-trivial comments that you may want to get back to:
README.md the first line of comment is
pwa-deployed-versions/newspeaklanguage.org.version
build.sh
I also added the changes to the pull request - you should see 2 commits in the pull request.
No rush, let me know if we need to follow up more on anything,
Thanks, Milan
Hi Milan. I'm ok with moving forward on this. I found a couple of typos I missed last time, but they are trivial - happens to me all the time. Thanks for your patience.
Sorry if any messages were confusing. I closed the PR by accident, then reopened it.
Gilad,
A few comments inline on your 1. 2. 3.:
On Sat, Feb 24, 2024 at 2:04 PM Gilad Bracha @.***> wrote:
@.**** commented on this pull request.
In tool/build.sh https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1501683310 :
+cp ${PRIMORDIALSOUP}/newspeak/.ns out +# 7. Merge in Newspeak IDE dependencies +cp ./.ns ./.png out +# cp -R CodeMirror out # CodeMirror is copied during deploy + +cd ${NEWSPEAK}/out || exit 1 + +# ${PRIMORDIALSOUP}/out/ReleaseX64/primordialsoup \ +# ${PRIMORDIALSOUP}/primordialsoup/out/snapshots/WebCompiler.vfuel \ +# .ns \ +# .png \ +# not in 'out': .txt \ +# not needed, included in html both index.html for WebIDE and primordialsoup.html: CodeMirror/lib/codemirror.js \ +# not needed, included in html both index.html for WebIDE and primordialsoup.html: CodeMirror/addon/display/autorefresh.js \ +# RuntimeForHopscotchForHTML HopscotchWebIDE HopscotchWebIDE.vfuel \ +# RuntimeForElectron HopscotchWebIDE HopscotchElectronIDE.vfuel \
On second thought, I'd leave it for now. When working on the IDE, I find that the PWA is still inferior for a number of small but annoying reasons. I'll list them for the record:
- The absence of forward navigation. I may be forced to change the iDE to show the full Chrome header. Oddly, there doesn't seem to be an option to get the reduced header with a forward button. Otherwise, one might research to see if we can detect if we are running as PWA and the use our own navigation buttons.
There seems to be a way to show or not show the back and reload buttons in a PWA, as described here: https://stackoverflow.com/questions/64058232/how-to-add-back-and-refresh-button-to-pwa-title-bar However, a quick search does not seem to describe if there is a way to show the forward button. Would it perhaps be better not to show any navigation buttons at all, and rely on the Newspeak's internal navigation? (on the second though , after experiments, back button is really valuable). If you could describe the preferred functionality (which navigation buttons to show or none at all) I will look into if it can be done in a PWA.
- Different behavior on saving files (at least on mac - if I'm saving Foo.ns and it already exists (the common case) it will want to save Foo (1).ns rather than Foo, forcing me to edit the name back to the original.
FWIW, on Linux Chrome, both the PWA version and the browser version wants to save as Foo (1).ns if Foo.ns already exists. If I understand correctly, for you the browser overrites Foo.ns? If so, the behavior may be OS or browser specific
- Different behavior wrt exception handling in the IDE. I'm not sure what is going on, but failures in the IDE code often lead to the system refreshing the page rather than displaying the debugger.
It is clearly a problem if the PWA does not display the debugger. I need to replicate an exception in a PWA and a browser, and see how this behaves for me, then follow up. Would you have a tip on how to duplicate this? Simply forcing a MNU works fine as far as I can tell.
Thanks, Milan
— Reply to this email directly, view it on GitHub https://github.com/newspeaklanguage/newspeak/pull/125#discussion_r1501683310, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALGQ7GW5BQA2TJAYL6W67LYVI2VVAVCNFSM6AAAAABC7HD4F6VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTQOJZGU2TKMBTGU . You are receiving this because you authored the thread.Message ID: @.***>
Hi Milan.
I successfully built Newspeak on my machine using a slightly modified version of the new setup. Specifically, I changed the env vars to the directories I normally used. I laos had to change mkdir --parent to mkdir -p, becaue I am using an older shell. So, yes it works, and so we can remove the 'I think' clauses. I have yet to complete running a version from scratch though.
Also, to be clear, I mad ethe changes you suggeste late last night.
Wrt your comments above:
I have not found any way to get a forward button. However, I learned that (at least on a mc) cmd-forward arrow works (and cmd-backward arrow likewise). That is actually not bad. I'm not sure how this works elsewhere - I'll test on Windows soon. If you can check how this works on Linux and let me know?
Another option is to change things so the explicit Newspeak navigations controls show up if we are running the PWA. I think I can more or less reliably detect this (longish story - as long as we a re not in fullscreen mode, it seems to be easy). I don't recall if this might interact poorly with the displayed back button - I'd have to check that out. So maybe having the controls and using standalone display will work well - we'll see. It's less urgent f the keyboard shortcuts work.
As for the name a file is saved under: what I mean is the opposite. I want it to overwrite the existing file, but instead it wants to write a new file Foo (1).ns. SI have to edit the name by had before saving. I have not yet found a way to control this behavior. One possibility might be start using the showSaveFilePicker() API,which didn't exist when I wrote this file saving code. It's not clear if it would behave differently - but it is a pain, being async, so I haven't tried it yet.
Lastly, debugging. This isn't always a problem. User code tested in the IDE usuallly enters the debugger just fine. However, if you have crash in the IDE (or whatever your deployed app is) it's trickier. It didn't work for a long time. Eventually I did some hacky stuff that mostly worked in Electron, but is less reliable in the PWA. Might be related to the use of service workers. So maybe adding a button in some IDE menu that always crashes might provoke it.
All this alas, takes more time. I really want to focus on other stuff (like teaching AI about Newspeak and integrating it in the system) so we'll see.
Let me add some observations, for the record.
a. In Windows, one can navigate via the keyboard much as on the mac. Instead of cmd- you use alt-, so alt -> goes forward and alt - <- goes backwards (where -> and <- represent the keyboard arrow buttons). I assume something similar works on Linux - please let me know. If so, there is little urgency to change things. b. If one wanted to change things to have The Newspeak navigation controls, there are several issues that would need to be addressed. One is the interaction of the Newspeak and browser controls. Hiding the browser controls is not enough, because the key board shortcuts are there. Another is the fact that one can move the display from the PWA window to the browser, and that will replicate the current state of these controls in the new display, which is wrong. It's not obvious if these two issue can be fixed, and anyway, they are a bit tedious. So I'm not going to spend more time on this now.
Hi Gilad,
That sounds good.
I will make a change and a pull request removing the 'I think' and also change 'mkdir --parent' to 'mkdir -p' for others who may use an older shell.
But maybe I should wait for you to complete installing and running everything from scratch?
Thanks, Milan
On Sun, Feb 25, 2024 at 7:47 PM Gilad Bracha @.***> wrote:
Hi Milan.
I successfully built Newspeak on my machine using a slightly modified version of the new setup. Specifically, I changed the env vars to the directories I normally used. I laos had to change mkdir --parent to mkdir -p, becaue I am using an older shell. So, yes it works, and so we can remove the 'I think' clauses. I have yet to complete running a version from scratch though.
— Reply to this email directly, view it on GitHub https://github.com/newspeaklanguage/newspeak/pull/125#issuecomment-1963135802, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALGQ7BNU4OH45APQRTAUX3YVPLSRAVCNFSM6AAAAABC7HD4F6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRTGEZTKOBQGI . You are receiving this because you authored the thread.Message ID: @.***>
Gilad, notes inline:
On Sun, Feb 25, 2024 at 8:05 PM Gilad Bracha @.***> wrote:
Wrt your comments above:
I have not found any way to get a forward button. However, I learned that (at least on a mc) cmd-forward arrow works (and cmd-backward arrow likewise). That is actually not bad. I'm not sure how this works elsewhere
- I'll test on Windows soon. If you can check how this works on Linux and let me know?
Yes, I can confirm on Linux Chrome the Newspeak PWA also honours the Chrome keyboard shortcuts: Alt+LeftArrow goes back, Alt+RightArrow goes forward.
Regarding Firefox, it appears they dropped PWA support and are now relying on a plugin, which also needs some os-specific binaries, so I have not tried Firefox.
Another option is to change things so the explicit Newspeak navigations
controls show up if we are running the PWA. I think I can more or less reliably detect this (longish story - as long as we a re not in fullscreen mode, it seems to be easy). I don't recall if this might interact poorly with the displayed back button - I'd have to check that out. So maybe having the controls and using standalone display will work well - we'll see. It's less urgent f the keyboard shortcuts work.
I agree, the working keyboard shortcuts for 'back' and 'forward' navigation make showing a widget (or something like that) with Newspeak navigation controls when in PWA not too important at least from my selfish perspective.
One question is whether we should remove the 'back' and 'reload' widget from the PWA ribbon, and perhaps add a note in help, something like 'In PWA, you can navigate back in history by using the browser and OS specific keyboard shortcut: In Chrome, pressing the ModifierKey and LeftArrow to go back, ModifierKey and RightArrow to go forward. The ModifierKey is 'Cmd' on the Mac, 'Alt' on Windows and Linux'. This should be simple, l'd volunteer to make those changes if it feels somewhat valuable, let me know.
As for the name a file is saved under: what I mean is the opposite. I want it to overwrite the existing file, but instead it wants to write a new file Foo (1).ns. SI have to edit the name by had before saving. I have not yet found a way to control this behavior. One possibility might be start using the showSaveFilePicker() API,which didn't exist when I wrote this file saving code. It's not clear if it would behave differently - but it is a pain, being async, so I haven't tried it yet.
Yes, I understood you prefer to ovewrite the existing file without editing the name. I was just saying for me on Linux Chrome, if Foo.ns exists, both the browser and the PWA behave the same - they suggest Foo (1).ns in the dialog before saving.
Lastly, debugging. This isn't always a problem. User code tested in the IDE
usuallly enters the debugger just fine. However, if you have crash in the IDE (or whatever your deployed app is) it's trickier. It didn't work for a long time. Eventually I did some hacky stuff that mostly worked in Electron, but is less reliable in the PWA. Might be related to the use of service workers. So maybe adding a button in some IDE menu that always crashes might provoke it.
Ok thanks for the explanation, I will try this (adding a button with no action or something like that to make it crash , and see if PWA and browser differ in behavior).
All this alas, takes more time. I really want to focus on other stuff (like teaching AI about Newspeak and integrating it in the system) so we'll see.
Interesting, thanks for sharing this.
Milan
— Reply to this email directly, view it on GitHub https://github.com/newspeaklanguage/newspeak/pull/125#issuecomment-1963146695, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALGQ7GCRWO27TPIJEBNKS3YVPNUXAVCNFSM6AAAAABC7HD4F6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRTGE2DMNRZGU . You are receiving this because you authored the thread.Message ID: @.***>
Gilad, I was reading and responding in the time-order, so there is a bit of overlap. Comments inline
On Mon, Feb 26, 2024 at 8:16 PM Gilad Bracha @.***> wrote:
Let me add some observations, for the record.
a. In Windows, one can navigate via the keyboard much as on the mac. Instead of cmd- you use alt-, so alt -> goes forward and alt - <- goes backwards (where -> and <- represent the keyboard arrow buttons). I assume something similar works on Linux - please let me know. If so, there is little urgency to change things.
Yes, as mentioned in the earlier response, on Linux Chrome, Alt+RightArrow and Alt+LeftArrow work to navigate forward and back. As those navigations are the same on Windows, per
https://support.google.com/chrome/answer/157179?hl=en&ref_topic=7437725&sjid=6662063252594173794-NC
we can be reasonably sure the 'Alt' version works on Windows as well.
b. If one wanted to change things to have The Newspeak navigation controls,
there are several issues that would need to be addressed. One is the interaction of the Newspeak and browser controls. Hiding the browser controls is not enough, because the key board shortcuts are there. Another is the fact that one can move the display from the PWA window to the browser, and that will replicate the current state of these controls in the new display, which is wrong. It's not obvious if these two issue can be fixed, and anyway, they are a bit tedious. So I'm not going to spend more time on this now.
That makes sense.
Thanks, Milan
— Reply to this email directly, view it on GitHub https://github.com/newspeaklanguage/newspeak/pull/125#issuecomment-1965617426, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALGQ7HW5ZB6QNQPREBZGBTYVUXW3AVCNFSM6AAAAABC7HD4F6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRVGYYTONBSGY . You are receiving this because you authored the thread.Message ID: @.***>
I forgot to mention another possibility, which is having the full fledged browser display. I have mixed feelings about that. It is ugly and occupies a lot of screen real estate - but it has the advantage(yet also disadvantage) of making it very clear that this is a web app, and making the navigation affordances obvious.
Perhaps best is what you suggest - get rid of the back button and refresh (i.e., replace minimal-ui with standalone as the manifest display) while adding help text to explain that.
Sounds good, I will make the manifest display 'standalone', add the help text, make sure it works, and make a pull request. Next day or two.
On Tue, Feb 27, 2024 at 4:13 PM Gilad Bracha @.***> wrote:
I forgot to mention another possibility, which is having the full fledged browser display. I have mixed feelings about that. It is ugly and occupies a lot of screen real estate - but it has the advantage(yet also disadvantage) of making it very clear that this is a web app, and making the navigation affordances obvious.
Perhaps best is what you suggest - get rid of the back button and refresh (i.e., replace minimal-ui with standalone as the manifest display) while adding help text to explain that.
— Reply to this email directly, view it on GitHub https://github.com/newspeaklanguage/newspeak/pull/125#issuecomment-1967599602, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALGQ7E7RX5NLT5CZPRQQKTYVZEA7AVCNFSM6AAAAABC7HD4F6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRXGU4TSNRQGI . You are receiving this because you authored the thread.Message ID: @.***>
…version numbers are managed and stored in git.
Only the PWA version for newspeaklanguage.org is kept in git. Others, such as the PWA version for localhost or any other URL is not kept in git, but users can choose to manage it locally outside of git.
README.md is significantly modified and expanded, it now describes more details about installing Newspeak from scratch, building, deploying and serving Newspeak applications.