Could you explain to some my questions?

magichim commented 6 years ago

First of all, I recently have a much curiosities about deno and researching it.

While I am tracing deno's commit and issues on github, I found your suggestion and this project repo.

I already have watched ryan's presentation on eu-jsconf and understand his goal of deno and new innovation on js and ts runtime environment.

Anyway, my questions is as follows.

ryan regrets binding c++ to v8 sticky so that he makes v8worker2 to avoid and secure as sendbox concept so that he uses protobuf to use message concept and then also he is now moving rust-v8worker in deno. is it right?
If a first question is true, deno's new rust v8worker is yours repo(this repo)? if not, his own makes new v8worker(rust ver)?.
your project(this pro) and v8worker2(original ryan's) is dispatcher like bottom picture?
What should I command and install to execute your project? please detaily, write test command exam on Readme.md or here.

I'm sorry to ask many questions. I want to understand deno's all of structure and how it works.

Thanks to read. : )

matiasinsaurralde commented 6 years ago

Hi @magichim! The new deno version is in progress and V8 is already integrated here (not as a separate repo which was the case for Go): https://github.com/ry/deno/blob/3b595253a2e9f8badc416f85d0b09bf48f344634/src/deno.cc#L239-L245 This project might still be useful for deno or projects that require a similar architecture (in-memory message passing between V8 and some other language, isolation, etc.). The architecture is exactly as you describe. I'm still tweaking this project but the instructions should be as follows:

Clone the repo, cd into the repo directory.
Run ./build.py (this will fetch and build V8).
Run cargo test.

Let me know how it goes!

magichim commented 6 years ago

@matiasinsaurralde Thanks to reply to me.

I followed your explanation. but there are some problems.

when I command ./build.py I got the error message

Running hooks: 100% (28/28), done.        
Traceback (most recent call last):
File "./build.py", line 152, in <module>
main()
File "./build.py", line 88, in main
lib_fn = Rebuild()
File "./build.py", line 103, in Rebuild
assert os.path.exists(gn_path)
AssertionError

this message might be same before deno v8 fetch (execute build.py).

and then I try to type cargo test and I got the error message


error: failed to run custom build command for `v8worker2 v0.1.0 (file:///Users/songwon/CODE/rust-v8worker2)`
process didn't exit successfully: `/Users/songwon/CODE/rust-v8worker2/target/debug/build/v8worker2-a2bb42e8aea7757c/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'Couldn't find V8 via pkg-config', build.rs:7:9
note: Run with `RUST_BACKTRACE=1` for a backtrace.

warning: build failed, waiting for other jobs to finish... error: build failed



My local environment, MacOS and I already installed deno's demend like (protobuf, depot_tools...etc)
For detail explaining this situation, I attach a captured image.

<img width="1259" alt="2018-06-27 10 33 12" src="https://user-images.githubusercontent.com/5843652/41947970-93594e2a-79f5-11e8-8007-355bbdcd5552.png">

matiasinsaurralde commented 6 years ago

@magichim You're welcome. Looks like an issue with the V8 download/build script. Let me take a look and come back to you asap. I've just started to fix the CI for this project and the V8 build goes fine (using Ubuntu). Locally I run OS X.

matiasinsaurralde commented 6 years ago

Can you post the full log for the build.py step?

magichim commented 6 years ago

@matiasinsaurralde yes

SONGWONs-MacBook-Pro:rust-v8worker2 songwon$ ./build.py 
out_path None
Rebuilding V8
Fetching dependencies.

________ running 'git -c core.deltaBaseCacheLimit=2g clone --no-checkout --progress https://chromium.googlesource.com/v8/v8.git ./_gclient_v8_mvhSF2' in '.'
Cloning into './_gclient_v8_mvhSF2'...
remote: Sending approximately 301.48 MiB ...        
remote: Counting objects: 7676, done        
remote: Finding sources: 100% (16/16)           
remote: Total 569798 (delta 483722), reused 569790 (delta 483722)        
Receiving objects: 100% (569798/569798), 300.47 MiB | 12.12 MiB/s, done.
Resolving deltas:  59% (286891/483722)   
[0:01:00] Still working on:
[0:01:00]   v8
Resolving deltas:  69% (337364/483722)   
[0:01:10] Still working on:
[0:01:10]   v8
Resolving deltas:  79% (383825/483722)   
[0:01:20] Still working on:
[0:01:20]   v8
Resolving deltas:  84% (408288/483722)   
[0:01:30] Still working on:
[0:01:30]   v8
Resolving deltas:  88% (426403/483722)   
[0:01:40] Still working on:
[0:01:40]   v8
Resolving deltas:  91% (440466/483722)   
[0:01:50] Still working on:
[0:01:50]   v8
Resolving deltas:  97% (469212/483722)   
[0:02:00] Still working on:
[0:02:00]   v8
Resolving deltas: 100% (483722/483722), done.

[0:02:10] Still working on:
[0:02:10]   v8
Syncing projects:   0% ( 0/ 2) 
[0:02:12] Still working on:
[0:02:12]   v8
Syncing projects: 100% (24/24), done.                                   

________ running 'download_from_google_storage --no_resume --platform=darwin --no_auth --bucket chromium-clang-format -s v8/buildtools/mac/clang-format.sha1' in '.'
0> Downloading v8/buildtools/mac/clang-format...
Downloading 1 files took 7.237365 second(s)
Running hooks:  42% (12/28) gn_mac            
________ running 'download_from_google_storage --no_resume --platform=darwin --no_auth --bucket chromium-gn -s v8/buildtools/mac/gn.sha1' in '.'
0> Downloading v8/buildtools/mac/gn...
Downloading 1 files took 6.319702 second(s)
Running hooks:  50% (14/28) wasm_spec_tests
________ running 'download_from_google_storage --no_resume --no_auth -u --bucket v8-wasm-spec-tests -s v8/test/wasm-spec-tests/tests.tar.gz.sha1' in '.'
0> Downloading v8/test/wasm-spec-tests/tests.tar.gz...
0> Extracting 72 entries from v8/test/wasm-spec-tests/tests.tar.gz to v8/test/wasm-spec-tests/tests
Downloading 1 files took 6.851213 second(s)
Running hooks:  53% (15/28) closure_compiler
________ running 'download_from_google_storage --no_resume --no_auth -u --bucket chromium-v8-closure-compiler -s v8/src/inspector/build/closure-compiler.tar.gz.sha1' in '.'
0> Downloading v8/src/inspector/build/closure-compiler.tar.gz...
0> Extracting 4 entries from v8/src/inspector/build/closure-compiler.tar.gz to v8/src/inspector/build/closure-compiler
Downloading 1 files took 7.007753 second(s)
Running hooks:  89% (25/28) clang               
________ running '/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python v8/tools/clang/scripts/update.py' in '.'
Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Mac/clang-335091-1.tgz .......... Done.
Running hooks: 100% (28/28), done.        
Traceback (most recent call last):
  File "./build.py", line 152, in <module>
    main()
  File "./build.py", line 88, in main
    lib_fn = Rebuild()
  File "./build.py", line 103, in Rebuild
    assert os.path.exists(gn_path)
AssertionError

matiasinsaurralde commented 6 years ago

Do you see gn inside the depot_tools directory?

% file depot_tools/gn
depot_tools/gn: Bourne-Again shell script text executable, ASCII text

magichim commented 6 years ago

I just cloned depot_tools from github and gn file has codes as follows.

SONGWONs-MacBook-Pro:depot_tools songwon$ cat gn
#!/usr/bin/env bash
# Copyright 2013 The Chromium Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.

base_dir=$(dirname "$0")

PYTHONDONTWRITEBYTECODE=1 exec python "$base_dir/gn.py" "$@"

when I execute gn file,

Traceback (most recent call last):
  File "./gn.py", line 47, in <module>
    sys.exit(main(sys.argv))
  File "./gn.py", line 25, in main
    'gn', 'gn' + gclient_utils.GetExeSuffix())
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py", line 70, in join
    elif path == '' or path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'endswith'

matiasinsaurralde commented 6 years ago

I see, I think you shouldn't call it directly, try setting the depot_tools path in a manual way, like this:

./build.py --rebuild --depot_tools_path $PWD/depot_tools

or

./build.py --depot_tools_path $PWD/depot_tools

or

./build.py --depot_tools_path path/to/depot_tools

magichim commented 6 years ago

@matiasinsaurralde I succeeded fetching v8 by using ./build.py --rebuild --depot_tools_path

but cargo test I got error as follows.

  Compiling v8worker2 v0.1.0 (file:///Users/songwon/CODE/rust-v8worker2)
warning: foreign function is never used: `worker_set_rust_object`
  --> src/binding.rs:22:5
   |
22 |     pub fn worker_set_rust_object(w: *mut worker, p: *mut wrk);
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: #[warn(dead_code)] on by default

error[E0308]: mismatched types
  --> src/lib.rs:25:42
   |
25 |     let mut worker = worker::Worker::new(_recv_cb);
   |                                          ^^^^^^^^ expected struct `std::boxed::Box`, found struct `bytes::Bytes`
   |
   = note: expected type `fn(bytes::Bytes) -> std::boxed::Box<bytes::Bytes>`
              found type `[closure@src/lib.rs:22:20: 24:6]`

error[E0308]: mismatched types
  --> src/lib.rs:26:17
   |
26 |     worker.load("code.js".to_string(), "V8Worker2.send(new ArrayBuffer(10))".to_string());
   |                 ^^^^^^^^^^^^^^^^^^^^^
   |                 |
   |                 expected &str, found struct `std::string::String`
   |                 help: consider borrowing here: `&"code.js".to_string()`
   |
   = note: expected type `&str`
              found type `std::string::String`

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0308`.
error: Could not compile `v8worker2`.
warning: build failed, waiting for other jobs to finish...
error: build failed

I don't know well about rust syntax, it looks like v8worker problem. right?

matiasinsaurralde commented 6 years ago

Very cool, yes, just found these errors, I broke master in my last changes. I'm fixing it now, sorry!

magichim commented 6 years ago

thanks, If you fix, comment on here please :)

matiasinsaurralde commented 6 years ago

I've fixed it and updated the callback test, consider running cargo test -- --nocapture. Will continue fixing the CI for this project. You should see:

running 1 test
Getting data from V8, length is 10
test test_wrapper ... ok

That's the buffer received from V8.

magichim commented 6 years ago

Thanks! From now on I research about this project. I'm not familiar rust lang.

Could you give me some hint about cargo test working process?

and Can you add some files in gitignore made after v8fetching (like v8 forder, .gclient_entries ..etc)

matiasinsaurralde commented 6 years ago

Cool, this project has only one very simple test, I might extend this in the future but the functionality is pretty simple. You can define tests using #[test], you can check src/lib.rs and scroll down to test_wrapper. That's the only test that's currently available:

#[test]
fn test_wrapper() {
    let mut _h = new_handler();
    // This will initialize V8:
    _h.init();
    // Implement a callback, accepts bytes and returns boxed bytes (in this way we can safely pass the data to the V8/C side, however this might leak memory at some point, I'm still researching the best way of freeing up this memory after V8 receives this):
    let cb = |incoming_data: bytes::Bytes| -> Box<bytes::Bytes> {
        println!("Getting data from V8, length is {}", incoming_data.len());
        // Break if the buffer doesn't have 10 bytes:
        assert!(incoming_data.len() == 10);
        // Prepare a reply containing "reply", this isn't useful at all but might be used for tests on the V8/C side:
        let data = Bytes::from(&b"reply"[..]);
        // Box the buffer and return:
        Box::new(data)
    };
    // Initialize a worker object using the above callback:
    let mut worker = worker::Worker::new(cb);
    // Call the "load" function, passing a "virtual" script name and calling the V8Worker2.send method:
    worker.load("code.js", "V8Worker2.send(new ArrayBuffer(10))".to_string());
}

This test calls V8Worker2.send(new ArrayBuffer(10)) in V8, which would trigger the Rust callback (V8 -> Rust) with an empty 10 bytes buffer. Then it checks that the received buffer contains the expected length. The --nocapture flag will let you see the Getting data from V8... message as it's hidden by default (during tests).

magichim commented 6 years ago

@matiasinsaurralde Thanks to you, I understood many things.

I compared original v8worker2(go lang) with this rust-v8worker.

However I have a one question.

How to use exactly worker load js file? I tested and understood (V8->Rust) call. but I don't know well usage of code.js file and where to locate code.js file for using?

I'm sorry to annoy you. thanks to read. :)

matiasinsaurralde commented 6 years ago

@magichim Hey, you're welcome, that's very inspiring! The worker load function doesn't read anything from the filesystem and the script name parameter is only used as a way of identifying this resource, see ScriptOrigin in the V8 documentation, also check the src/binding.cc file where this function is set:

int worker_load(worker* w, char* name_s, char* source_s) {
  Locker locker(w->isolate);
  Isolate::Scope isolate_scope(w->isolate);
  HandleScope handle_scope(w->isolate);

  Local<Context> context = Local<Context>::New(w->isolate, w->context);
  Context::Scope context_scope(context);

  TryCatch try_catch(w->isolate);

  Local<String> name = String::NewFromUtf8(w->isolate, name_s);
  Local<String> source = String::NewFromUtf8(w->isolate, source_s);

  ScriptOrigin origin(name);

  Local<Script> script = Script::Compile(source, &origin);

  if (script.IsEmpty()) {
    assert(try_catch.HasCaught());
    w->last_exception = ExceptionString(w, &try_catch);
    return 1;
  }

  Handle<Value> result = script->Run();

  if (result.IsEmpty()) {
    assert(try_catch.HasCaught());
    w->last_exception = ExceptionString(w, &try_catch);
    return 2;
  }

  return 0;
}

matiasinsaurralde commented 6 years ago

Also check this file in reno, it's just a personal experiment I'm running to learn Rust and try to solve some of the architecture and implementation details that Deno describes.

matiasinsaurralde / rust-v8worker2

Could you explain to some my questions? #2