support nodejs REPL console inside neo-blessed

[iurimatias]

Outline

Currently the embark dashboard (shown in embark run) has a log section and a input field to type commands. This input field is not very good as it doesn't support going back with a cursor to correct something, history is limited, there is no autocomplete, etc.. On the other hand embark console (available develop branch) is a full featured REPL.

The goal of this task is to support the same REPL console used in embark console in the embark dashboard. This likely means some changes will be needed in neo-blessed

Acceptance Criteria

• The embark dashboard in embark run, the input field should disappear, and the log section should have a REPL with the same features as the one embark console

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 800.0 DAI (800.0 USD @ $1.0/DAI) attached to it.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
  • Want to chip in? Add your own contribution here.
  • Questions? Checkout Gitcoin Help or the Gitcoin Slack
  • $24,187.14 more funded OSS Work available on the Gitcoin Issue Explorer

[StatusSceptre]

Hey @pinkiebell - @lastmjs applied first here, but if you guys want to work together, we'd be happy to split the bounty for this. Lmk what you would like to do and what will work best, we're still experimenting with the Gitcoin stuff...

[pinkiebell]

@StatusSceptre No problem 👍 @lastmjs has the favour, let me know if you need help 😃

[vs77bb]

Both great candidates! @lastmjs glad to see you here - don't hesitate to reach out with questions as you progress 🙂

[lastmjs]

Sounds good, I just started familiarizing myself with the codebase, I should make significant progress tomorrow, and don't worry I'll be getting on Slack soon

[lastmjs]

Update: I've gotten rid of the console input, and I have the node REPL outputting to the Log area, though jankily.

[lastmjs]

I want to get some thoughts on what's going on. I don't know if it's the best idea to get rid of the console section and consolidate it into the log section. The reason is that logs can come from various places in the application at various times. If the repl is inside of the log container, then what you're currently typing could be written over at any time by a random log. If we separate the log and console sections, but have the console output go into the log section like the original setup, then we get around that problem. Any objections to this?

[lastmjs]

To get this to work, I think I'm going to need to use a blessed.terminal, and to get that to work I need the dependency pty.js. Unfortunately, pty.js has some V8 compatibility issues (it doesn't work with my version of Node.js 10). I installed the latest Node.js 8 and the V8 compatability issues are gone.

Track this issue: https://github.com/chjj/pty.js/issues/195

[pinkiebell]

@lastmjs If you pass a handler: function(){} to blessed.Terminal, then you don't need pty.js 👍 .

[iurimatias]

@lastmjs I need to consult the team about this

[iurimatias]

@lastmjs alright it makes sense.

[lastmjs]

Alright, things are working pretty well. Looks like I just need to clean up and figure out styling, I'll get a PR up today to start looking at

[lastmjs]

I'm trying to style the terminal, but it's having some major issues. If I try to put a border directly on the terminal, the following errors get thrown:

pid 2783 listening on /home/lastmjs/development/embark_demo/.embark/embark.ipc

<--- Last few GCs --->

[2783:0x29e7c70]     5329 ms: Mark-sweep 1398.2 (1425.0) -> 1397.8 (1425.5) MB, 187.6 / 0.0 ms  (average mu = 0.118, current mu = 0.025) allocation failure scavenge might not succeed
[2783:0x29e7c70]     5524 ms: Mark-sweep 1398.5 (1425.5) -> 1398.1 (1426.0) MB, 194.0 / 0.0 ms  (average mu = 0.062, current mu = 0.002) allocation failure scavenge might not succeed

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x2af2979041bd]
    1: StubFrame [pc: 0x2af297915610]
Security context: 0x33927891e6c9 <JSObject>
    2: new Terminal [0x3c8358cb2f19] [/home/lastmjs/development/embark/node_modules/term.js/src/term.js:~182] [pc=0x2af298033a3a](this=0x3c8358cb2d21 <Terminal map = 0x3c21858fa129>,options=0x3c8358cb2f99 <Object map = 0x3c21858ffb39>)
    3: ConstructFrame [pc: 0x2af29790f147]
    4: StubFrame [pc: 0x2af29800d2c0]
    5...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x8b5f80 node::Abort() [node]
 2: 0x8b5fcc  [node]
 3: 0xab730e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xab7528 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xea5152  [node]
 6: 0xeb10aa v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
 7: 0xeb1a14 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 8: 0xeb4345 v8::internal::Heap::AllocateRawWithRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [node]
 9: 0xe7c824 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [node]
10: 0x111e1de v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [node]
11: 0x2af2979041bd 
Aborted (core dumped)

If I try to make the terminal the child of a box, the screen goes blank and I can't get out of it without quitting the terminal. Anyone know how to get around these issues?

[lastmjs]

I tracked the issue I was having above down to the height property of the terminal...seems like a height of 4% is causing the error. 5% or 10% is fine, but 4% causes the error

[lastmjs]

How necessary is it that the output from the repl go into the log section? The problem is differentiating, inside of the repl's writable stream, between input from the user that should be written to the one line terminal, or the actual output of the repl that should go into the log section. It gets complicated because of chunking and things like autocomplete that is an exception to the repl output (it doesn't run through the same output function that allows me to capture the output). Also, styling the terminal to be one line and never have any ill-effects from different terminal sizes and such will probably be difficult.

So, if there is any way that we could have a log section and a terminal section, with separate inputs and outputs, that would be ideal on the development side. Thoughts?

[iurimatias]

Hi @lastmjs

We discussed & analyzed this and we feel that separating the logs is not the way to go as it would cause other problems. However we experimented with this and it seems that when the input is at the right height, all the issues reported pretty much disappear. for e.g setting it to 5% for a good height pretty much fixes all issues including the overflow, except the autocomplete (if there are multiple options, they wouldn't show), however that's OK. The ideal % seems to vary depending on the height of the terminal, so a possible solution is to detect window resizes, check the height, and then automatically set a new height for the input based on the based (can be checked with something like window-size. Let us know what you think

[iurimatias]

Another solution involves changing/improving the "css" layout of the dashboard so the input size is always correct.

[andremedeiros]

Hey @lastmjs, I have a minimal proof of concept that'll show you something that could make your life a lot easier. Effectively we want to keep the size of the REPL box constant. This snippet got me there (contracts is fixed, logs is flexible, REPL is fixed.) Try running this and resizing your terminal to see what I mean.

const blessed = require('neo-blessed');

const screen = blessed.screen({smartCSR: true, dockBorders: true});
screen.title = 'Embark UI';

const contractsBox = blessed.box({
  position: {
    left: 0,
    top: 0
  },
  width: '100%',
  height: 15,
  title: 'OMG',
  content: 'contracts',
  border: {
    type: 'line'
  },
  style: {
    border: {
      fg: 'green'
    }
  }
});

const logsBox = blessed.box({
  position: {
    left: 0,
    top: 16,
  },
  content: 'logs',
  width: '100%',
  height: '100%-18',
  border: {
    type: 'line'
  },
  style: {
    border: {
      fg: 'green'
    }
  }
});

const replBox = blessed.box({
  position: {
    left: 0,
    top: '100%-3'
  },
  content: 'repl',
  height: 3,
  width: '100%',
  border: {
    type: 'line'
  },
  style: {
    border: {
      fg: 'green'
    }
  }
});

screen.append(contractsBox);
screen.append(logsBox);
screen.append(replBox);

screen.render();

[lastmjs]

Sweet, I'll get back to this on Monday

On Fri, Sep 14, 2018, 3:31 PM André Medeiros notifications@github.com wrote:

Hey @lastmjs https://github.com/lastmjs, I have a minimal proof of concept that'll show you something that could make your life a lot easier. Effectively we want to keep the size of the REPL box constant. This snippet got me there (contracts is fixed, logs is flexible, REPL is fixed.) Try running this and resizing your terminal to see what I mean.

const blessed = require('neo-blessed'); const screen = blessed.screen({smartCSR: true, dockBorders: true});screen.title = 'Embark UI'; const contractsBox = blessed.box({ position: { left: 0, top: 0 }, width: '100%', height: 15, title: 'OMG', content: 'contracts', border: { type: 'line' }, style: { border: { fg: 'green' } } }); const logsBox = blessed.box({ position: { left: 0, top: 16, }, content: 'logs', width: '100%', height: '100%-18', border: { type: 'line' }, style: { border: { fg: 'green' } } }); const replBox = blessed.box({ position: { left: 0, top: '100%-3' }, content: 'repl', height: 3, width: '100%', border: { type: 'line' }, style: { border: { fg: 'green' } } }); screen.append(contractsBox);screen.append(logsBox);screen.append(replBox); screen.render();

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/embark-framework/embark/issues/768#issuecomment-421489954, or mute the thread https://github.com/notifications/unsubscribe-auth/AGrSjxnp_vqnXHz5Vy6IPCXo7F3HKGMtks5ubCBMgaJpZM4WVOdT .

[gitcoinbot]

@lastmjs Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] warning (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[lastmjs]

I'll be getting back to this as soon as I can, which should be today

On Thu, Sep 20, 2018 at 10:17 AM Gitcoin.co Bot notifications@github.com wrote:

@lastmjs https://github.com/lastmjs Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• warning (3 days)
• escalation to mods (6 days)

Funders only: Snooze warnings for 1 day https://gitcoin.co/issue/embark-framework/embark/768/1082?snooze=1 | 3 days https://gitcoin.co/issue/embark-framework/embark/768/1082?snooze=3 | 5 days https://gitcoin.co/issue/embark-framework/embark/768/1082?snooze=5 | 10 days https://gitcoin.co/issue/embark-framework/embark/768/1082?snooze=10 | 100 days https://gitcoin.co/issue/embark-framework/embark/768/1082?snooze=100

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/embark-framework/embark/issues/768#issuecomment-423243675, or mute the thread https://github.com/notifications/unsubscribe-auth/AGrSj9_L74SpWnjP60YQ1BGEfGVzOGbJks5uc7-JgaJpZM4WVOdT .

-- Jordan Last

819 N 500 E Logan, UT 84321 801-709-1860

https://www.linkedin.com/in/lastmjs https://github.com/lastmjs

If you don't want me emailing you again for whatever reason, just reply and let me know, thanks!

[andremedeiros]

@lastmjs looking forward to it!

[lastmjs]

@andremedeiros Great, I see what your demo is trying to do. Though it seems easy to lose the log area entirely like that. Is that okay? Would it be better to make everything else flexible, and just keep the repl box fixed in height?

[andremedeiros]

Keeping everything flexible is what got us in this mess in the first place ;-)

I can't speak definitely, but I think this should be the way to go. @iurimatias, thoughts?

[iurimatias]

IMO, for now let's keep it flexible but with the repl box fixed in height.

[iurimatias]

@lastmjs and to clarify, by "fixed" I mean, it can still be 'updated on the fly' if needed by terminal resize events.

[lastmjs]

Finally back at it, sorry for the delay. I just updated the PR.

[gitcoinbot]

@lastmjs Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] warning (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[lastmjs]

Looks like we're getting close, just a few small issues to resolve

[gitcoinbot]

@lastmjs Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] warning (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[lastmjs]

Looking into finishing up today

[andremedeiros]

Hey @lastmjs,

I gave the branch a run, and the command line feels sooooo crisp! Definitely a step in the right direction.

I am, however, having some issues with the UI re-orienting itself on my terminal, specifically on 186 cols and 97 lines, but can replicate it in other setups.

I think it could be helpful to jump into a call and maybe show you what I mean.

Damn, this feels so close I can almost taste it!

[lastmjs]

What time works for you? I'm at SF Bockchain Week most days this week and wasn't planning on bringing my laptop, but perhaps we can get on a call at night Pacific Time or early morning Pacific Time?

[andremedeiros]

I'm on EST. Tomorrow I have the whole day, from 10am EST to around 4pm EST if that helps.

If you'd prefer skipping the laptop, I'd effectively try your code with non standard sized terminals. Portrait instead of landscape, resize the window on both height and width to see some weird artifacts going on, maybe even reduce font size. I've tried all of these, and the call would be to demonstrate this.

[lastmjs]

Alright, I'm traveling tomorrow, I'll be back to normal life on Monday. Are the artifacts you're seeing disappearing borders and such? I've seen at least one example of that. I can do a call any time on Monday, or a screen capture/video would help as well

On Tue, Oct 9, 2018 at 3:57 PM André Medeiros notifications@github.com wrote:

I'm on EST. Tomorrow I have the whole day, from 10am EST to around 4pm EST if that helps.

If you'd prefer skipping the laptop, I'd effectively try your code with non standard sized terminals. Portrait instead of landscape, resize the window on both height and width to see some weird artifacts going on, maybe even reduce font size. I've tried all of these, and the call would be to demonstrate this.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/embark-framework/embark/issues/768#issuecomment-428367071, or mute the thread https://github.com/notifications/unsubscribe-auth/AGrSjy06RhH7s1lfes72XnAwVIbNJl9Jks5ujRu6gaJpZM4WVOdT .

-- Jordan Last

819 N 500 E Logan, UT 84321 801-709-1860

https://www.linkedin.com/in/lastmjs https://github.com/lastmjs

If you don't want me emailing you again for whatever reason, just reply and let me know, thanks!

[lastmjs]

I think I could use some clarity. Do we have a design for portrait versus landscape, or a responsive design that we will be following? For example, do we want the Environment, Status, and Available Services boxes to stack vertically under the Contracts box when the screen gets too narrow? Or do we want those boxes to be scrollable when the screen is too narrow?

[lastmjs]

Any clarity here?

[StatusSceptre]

@andremedeiros or @iurimatias any update on this one for @lastmjs?

[iurimatias]

@lastmjs can we chat on gitter?

[gitcoinbot]

Issue Status: 1. Open 2. Cancelled

---

The funding of 800.0 DAI (800.0 USD @ $1.0/DAI) attached to this issue has been cancelled by the bounty submitter

• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $45,974.14 more funded OSS Work available on the Gitcoin Issue Explorer