maciejhirsz / logos

Create ridiculously fast Lexers
https://logos.maciej.codes
Apache License 2.0
2.89k stars 118 forks source link

Add an ability to use smart pointers for source #341

Open InfiniteCoder01 opened 1 year ago

InfiniteCoder01 commented 1 year ago

This might be useful for #159, #324 and #340. I don't know anything about proc-macros in rust, but I'll try to implement it.

InfiniteCoder01 commented 1 year ago

I did not realize that this feature is there, we need docs for proc macros. But it's expecting type to be something that we put behind a reference, which is not the case with Rc or Arc. Changing the title

jeertmans commented 1 year ago

Hello @InfiniteCoder01, could you provide an example code showcasing a basic use case?

InfiniteCoder01 commented 1 year ago

Example of this that I can think of is quite complex, but here it is: When using logos in combination with codespan_reporting and writing a compiler for a language, that is like rust - uses mod somemodule; to mark somemodule.rs or somemodule/mod.rs as files to parse & link. Especially useful with multithreading. You are parsing a file, lexer is a stream of tokens, you encounter mod somemodule;. (Normally you would put it into AST but in my case AST is more like LLVM IR). You need to search for module file and add it to codespan_reporting Files. If those files where borrowed by lexer, it would be impossible to add a file immediately. When using Arc, we can read this file and put it into this database immediately, perhaps even start a separate thread, parsing it. This explanation is quite complex, but I hope it made it clearer.

InfiniteCoder01 commented 1 year ago

For now, we can use a workaround:

let source = file.source().clone(); // File is codespan_reporting::files::SimpleFile<String, Rc<str>>
let mut lexer = lexer::Token::lexer(&source);
axelkar commented 3 months ago

Can't you just deref Arc to &T? and use &T as the source?