search

racer-rust / racer

Rust Code Completion utility
MIT License
3.36k stars 278 forks source link

support for 3rd party crates from crates.io mirrors? #513

Open lilydjwg opened 8 years ago

lilydjwg commented 8 years ago

Hi, I heard racer supports completing for third party crates. That's great! However, it hard codes the github.com index. I'm using a crates.io mirror (index = "git://crates.mirrors.ustc.edu.cn/index" in ~/.cargo/config) because I'm in China and the network speed to GitHub is very slow.

Would racer read cargo's config file to find out which index to use?

phildawes commented 8 years ago

Hello, Racer's cargo support is a bit hacky, and it does hardcode the 'github' directory. The github specific code is in here, if you fancy making a patch: (search for github) https://github.com/phildawes/racer/blob/master/src/racer/cargo.rs

What does your .cargo directory structure look like? Thanks!

lilydjwg commented 8 years ago

This is what tree -L 4 ~/.cargo outputs. I've removed a lot of similar entries since it's too long.

/home/lilydjwg/.cargo
├── config
├── git
│   ├── checkouts
│   │   ├── bytes-b21aba8e191ae1b0
│   │   │   ├── 7edb577d0a
│   │   │   └── master
│   │   ├── cargo-0c67b1a5f6ce0f88
│   │   │   └── master
│   │   ├── cargo-2f7f63790bcc5cbc
│   │   │   └── master
│   │   ├── nix-rust-8d1a824a13ec5246
│   │   │   ├── ad87c3bdc9
│   │   │   ├── f37e45628e
│   │   │   └── master
│   └── db
│       ├── bytes-b21aba8e191ae1b0
│       │   ├── config
│       │   ├── description
│       │   ├── FETCH_HEAD
│       │   ├── HEAD
│       │   ├── hooks
│       │   ├── info
│       │   ├── objects
│       │   └── refs
│       ├── cargo-0c67b1a5f6ce0f88
│       │   ├── config
│       │   ├── description
│       │   ├── FETCH_HEAD
│       │   ├── HEAD
│       │   ├── hooks
│       │   ├── info
│       │   ├── objects
│       │   └── refs
└── registry
    ├── cache
    │   ├── crates.mirrors.ustc.edu.cn-4496af6807a54617
    │   │   ├── advapi32-sys-0.1.2.crate
    │   │   ├── aho-corasick-0.5.1.crate
    │   │   ├── ansi_term-0.7.2.crate
    │   │   ├── bitflags-0.1.1.crate
    ├── index
    │   ├── crates.mirrors.ustc.edu.cn-4496af6807a54617
    │   │   ├── cm
    │   │   ├── co
    │   │   ├── config.json
    │   │   ├── cp
    │   └── github.com-88ac128001ac3a9a
    │       ├── co
    │       ├── config.json
    │       ├── cp
    └── src
        ├── crates.mirrors.ustc.edu.cn-4496af6807a54617
        │   ├── advapi32-sys-0.1.2
        │   ├── aho-corasick-0.5.1
        │   ├── ansi_term-0.7.2
        │   ├── bitflags-0.1.1
        └── github.com-88ac128001ac3a9a
            ├── advapi32-sys-0.1.2
            ├── aho-corasick-0.3.4
            ├── aho-corasick-0.4.0
bruno-medeiros commented 8 years ago

BTW, this issue also seems to manifest when Cargo.lock references a specific Git revision, for example:

[[package]]
name = "clipboard"
version = "0.1.2"
source = "git+https://github.com/aweinstock314/rust-clipboard#4fe9892d7bf8bac075aa60493c6eee7a83e24b7f"

which also makes the crate source go to .cargo/git/checkouts.

Perhaps now cargo metadata should be used to fix this? The metadata command will provide the correct file system location of each crate root, so Racer doesn't need to replicate the logic to figure it out.

bruno-medeiros commented 8 years ago

(I might be up for working on this when I have the time, but it won't be anytime soon)

jwilm commented 8 years ago

@bruno-medeiros do you know if the cargo metadata functionality is available from a crate? execing a process each time a crate lookup is done would hurt performance..

bruno-medeiros commented 8 years ago

I don't know if it is, you'd have to ask the Cargo devs. But I don't think executing the metadata functionality inside Racer would make that much more of an improvement. (the exec overhead is not that bad?)

Note that you only request cargo metadata once per Racer operation. You request it for the crate where the Racer operation is invoked, and the metadata will give you the source info for all the dependent crates.

To improve on that (caching the metadata for example), would require Racer running on deamon mode.

jwilm commented 8 years ago

Cargo appears to be on crates.io, and the metadata operation seems to rely on resolve_dependencies. Maybe this feature could use that directly.

If the metadata resolution is fast enough, it might not be worth caching. Cache invalidation seems difficult here since it would require knowledge of cargo's metadata resolution.

jwilm commented 8 years ago

I looked into using the cargo metadata machinery to resolve dependencies. There is a dependency resolution step involving updating the registry information, and updating git repositories. From what I can tell, there is no way to use cargo internals to return the dependency paths on disk without this network activity, and it is slow even in the case of a no-op (think 1-2 seconds).

My current idea to resolve this would be writing a more formal dependency analysis crate that is knowledgeable about cargo/multirust/rustup.rs and can do the local path resolution quickly, but that's more work than simply dropping in cargo.

bruno-medeiros commented 8 years ago

You can cache the cargo metadata information based on the timestamps of Cargo.toml and Cargo.lock. Basically only run cargo metadata if the modification time of any of those two files changed.

norru commented 8 years ago

Has this feature being released? I've got latest RustDT, rustup and racer, Ctrl+Click on symbol and autocomplete only work on 1) current project 2) rustc src. External crates don't get resolved, for instance: ctrl+click on

use super::timing::Duration;

here's the Rust Tools Log:

>> Running: /home/nico/.cargo/bin/racer find-definition 3 19 /home/nico/Projects/3rdParty/amethyst/src/engine/state.rs
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
ERROR:racer::cargo: ERROR!: Error { repr: Os { code: 2, message: "No such file or directory" } } .cargo/registry/src/github.com-88ac128001ac3a9a/racer-1.2.9/src/racer/cargo.rs 435
END
  ^^^ Terminated, exit code: 0 ^^^

I can provide more detailed or specific test results. It's just that I need to know whether I'm chasing a wild goose, because feature is not implemented, before doing so.

Thanks!

hawnzug commented 7 years ago

I have the same problem with @lilydjwg when using crate.io mirror(git://mirrors.ustc.edu.cn/crates.io-index) in China. Here my tree output under .cargo/registry/index

tree -L 1
.
├── crates.mirrors.ustc.edu.cn-245570943fb24b9d
├── crates.mirrors.ustc.edu.cn-e2f007c923a8536a
├── github.com-1ecc6299db9ec823
├── github.com-88ac128001ac3a9a
└── mirrors.ustc.edu.cn-61ef6e0cd06fb9b8

5 directories, 0 files

Removing these hardcoded validation codes and simply pushing the path temporarily works for me, but it may cause some conflicts or misleading results if some crates were in both index(I'm not sure because now my crates is completely dependent on the mirror so this workaround seems to be working fine).

As @lilydjwg suggests, analyzing cargo's config file to find out which index to use may be a better approach.

xzn commented 6 years ago

A related issue is that 'patch' and 'replace' section of Cargo.toml are ignored. Using 'cargo metadata' would fix this as well, in addition to getting the right version of the crate requested. I've looked into src/racer/cargo.rs however the api seems limited though, not sure how to go about fixing this.

Another issue with 'cargo metadata' is that it makes network requests to update index and download missing crates, I've tried passing -Zoffline but this just causes error when crates are missing.

kngwyu commented 6 years ago

Fixed in my branch, tough this implementation depends on internet:fearful:

ksqsf commented 6 years ago

racer 2.0.14. Still affected. A simple workaround is as follows in case anybody reaches here.

cd ~/.cargo/registry/src
ln -s mirrors.ustc.edu.cn-* github.com-
sidgwick commented 6 years ago

@ksqsf thanks, it works for me.

huangjj27 commented 5 years ago

For windows user, the mklink command also solve the problem

cd .cargo\registry\src
mklink github.com- mirrors.ustc.edu.cn-*
sbwtw commented 5 years ago

@huangjj27 Should be using /D option to create a dir link (default is file link)

mklink /D github.com- mirrors.ustc.edu.cn-*
WangPengJu commented 4 years ago

still can't work i try use 

cd ~/.cargo/registry/src
ln -s mirrors.ustc.edu.cn-* github.com-

but still can't work