search

realm / jazzy

Soulful docs for Swift & Objective-C
https://realm.io
MIT License
7.35k stars 413 forks source link

Handle projects with no documentation comments #94

Closed juangamnik closed 9 years ago

juangamnik commented 10 years ago

Using jazzy v0.0.14 and ruby 2.1.5 on Yosemite installed via Homebrew I get a JSON parsing error and then a whole bunch of JSON code printed to console.

my-swift-prj$ jazzy
Running xcodebuild -dry-run
parsing Xyz.swift (1/n)
Xyz.swift is 40% documented
[...]
building site
/usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse': 399: unexpected token at ',  (JSON::ParserError)
[...]
'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.14/lib/jazzy/sourcekitten.rb:162:in `parse'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.14/lib/jazzy/doc_builder.rb:94:in `build_docs_for_sourcekitten_output'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.14/lib/jazzy/doc_builder.rb:51:in `build'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.14/bin/jazzy:15:in `<top (required)>'
    from /usr/local/bin/jazzy:23:in `load'
    from /usr/local/bin/jazzy:23:in `<main>'

Ideas?

segiddins commented 10 years ago

Can you please share an example project that demonstrates this issue (or the json at the very least)? Otherwise it will be very hard to track down what's happening

juangamnik commented 10 years ago

Oh sorry for the delay. I would like to share an example project with you, but I can just upload PNG, GIF, and JPG here. Hence here is a link to the project: https://oc.kingsware.de/public.php?service=files&t=4fad92d0f2bd5d54adef92e6e4fd99ba

jpsim commented 10 years ago

@juangamnik what are you hoping to document in that project? There are no documentation comments. You can learn more about documenting Swift projects here: http://nshipster.com/swift-documentation/

In the meantime, we'll update jazzy to gracefully handle cases in which there are no documentation comments.

jpsim commented 10 years ago

sourcekitten output for this project is

[
{
  "key.offset" : 0,
  "key.doc.coverage" : 0,
  "key.doc.undocumented" : 6,
  "key.diagnostic_stage" : "source.diagnostic.stage.swift.parse",
  "key.length" : 2358,
  "key.substructure" : [

  ],
  "key.doc.documented" : 0
},
]

which is invalid JSON. sourcekitten is outputting this because GameScene.swift is completely undocumented.

segiddins commented 10 years ago

oh JSON...

juangamnik commented 10 years ago

@jpsim This is just a minimal project, which shows that jazzy exits with an error if a class is undocumented (didn't know that this causes the error, because another project that had no documentation worked quite well and just gave me empty documentation pages). In my real project I have documented classes, too, of course. But if I add a new class, it is not documented in the first place and the whole jazzy-execution fails and I get no docs at all. But as you said, this is not the expected behavior.

Thank you very much for your help.

orta commented 10 years ago

Yeah, I'd expect to see this case a bunch ;)

segiddins commented 10 years ago

The offending line is https://github.com/jpsim/sourcekitten/blob/master/sourcekitten/main.swift#L681

jpsim commented 10 years ago

@segiddins not quite, it's more that the second file should actually print out the same as the first file (they're both undocumented):

{
  "key.offset" : 0,
  "key.doc.coverage" : 0,
  "key.doc.undocumented" : 6,
  "key.diagnostic_stage" : "source.diagnostic.stage.swift.parse",
  "key.length" : 2358,
  "key.substructure" : [

  ],
  "key.doc.documented" : 0
}
jpsim commented 9 years ago

Thanks for reporting this issue, @juangamnik, a fix has now been pushed to master.

juangamnik commented 9 years ago

Thanks for fixing. Now I get an error during updated with sudo gem update, but jazzy seems to be updated to 0.0.16 now (does this release contain the fix?). But then during building the site I get another JSON parse error:

/usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse': 399: unexpected token at '] (JSON::ParserError)
'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.16/lib/jazzy/sourcekitten.rb:142:in `parse'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.16/lib/jazzy/doc_builder.rb:100:in `build_docs_for_sourcekitten_output'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.16/lib/jazzy/doc_builder.rb:53:in `build'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.16/bin/jazzy:15:in `<top (required)>'
    from /usr/local/bin/jazzy:23:in `load'
    from /usr/local/bin/jazzy:23:in `<main>'

During update this error occurred:

Updating jazzy
Fetching: rouge-1.7.4.gem (100%)
Successfully installed rouge-1.7.4
Fetching: sqlite3-1.3.10.gem (100%)
Building native extensions.  This could take a while...
Successfully installed sqlite3-1.3.10
Fetching: jazzy-0.0.16.gem (100%)
Successfully installed jazzy-0.0.16
Parsing documentation for jazzy-0.0.16
Installing ri documentation for jazzy-0.0.16
Installing darkfish documentation for jazzy-0.0.16
Parsing documentation for rouge-1.7.4
Installing ri documentation for rouge-1.7.4
Installing darkfish documentation for rouge-1.7.4
/usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/class_module.rb:173:in `aref_prefix': missing aref_prefix for RDoc::SingleClass (NotImplementedError)
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/class_module.rb:181:in `aref'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/template/darkfish/class.rhtml:47:in `block in generate_class'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/erb.rb:850:in `eval'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/erb.rb:850:in `result'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:724:in `template_result'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:703:in `block in render_template'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:698:in `open'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:698:in `open'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:698:in `render_template'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:351:in `generate_class'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:371:in `block in generate_class_files'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:368:in `each'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:368:in `generate_class_files'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/generator/darkfish.rb:245:in `generate'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:137:in `block in document'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:134:in `chdir'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:134:in `document'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:196:in `generate'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:56:in `block in generation_hook'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:55:in `each'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rdoc/rubygems_hook.rb:55:in `generation_hook'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/dependency_installer.rb:400:in `call'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/dependency_installer.rb:400:in `block (2 levels) in install'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/dependency_installer.rb:399:in `each'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/dependency_installer.rb:399:in `block in install'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/dependency_installer.rb:353:in `in_background'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/dependency_installer.rb:398:in `install'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/commands/update_command.rb:206:in `update_gem'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/commands/update_command.rb:220:in `block in update_gems'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/commands/update_command.rb:219:in `each'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/commands/update_command.rb:219:in `update_gems'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/commands/update_command.rb:98:in `execute'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/command.rb:305:in `invoke_with_build_args'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/command_manager.rb:167:in `process_args'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/command_manager.rb:137:in `run'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/rubygems/gem_runner.rb:54:in `run'
    from /usr/local/bin/gem:21:in `<main>'
jpsim commented 9 years ago

We haven't pushed an update to rubygems since making the fix. I'll do that for you now.

jpsim commented 9 years ago

@juangamnik I just pushed 0.0.17 to rubygems. Let me know if that works for you.

juangamnik commented 9 years ago

I updated to 0.0.17 and get the following error:

building site
/usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse': 399: unexpected token at ', (JSON::ParserError)
{
  "key.substructure" : [

  ],
  "key.offset" : 0,
  "key.diagnostic_stage" : "source.diagnostic.stage.swift.parse",
  "key.length" : 737
}

]
'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.17/lib/jazzy/sourcekitten.rb:174:in `parse'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.17/lib/jazzy/doc_builder.rb:100:in `build_docs_for_sourcekitten_output'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.17/lib/jazzy/doc_builder.rb:53:in `build'
    from /usr/local/lib/ruby/gems/2.1.0/gems/jazzy-0.0.17/bin/jazzy:15:in `<top (required)>'
    from /usr/local/bin/jazzy:23:in `load'
    from /usr/local/bin/jazzy:23:in `<main>'
jpsim commented 9 years ago

hmmm, the project you linked to works for me. Have you updated the project since you linked it? What version of Xcode do you have installed under /Applications/Xcode.app? What does xcode-select -p print out? What version of OS X?

juangamnik commented 9 years ago

Sorry. For the linked project it works now. But it was just a nearly empty one. Another bigger iOS-project works, too. But this is a custom cocoa touch framework project. Perhaps that is already the problem? I created it to separate model from view, in order to use the model in a playground. It works well (using a container workspace) and even the REPL works, when I debug my application in the simulator.

Xcode: Version 6.1.1 (6A2006) Yosemite: 10.10.2 (14C81f)

 $ xcode-select -p
/Applications/Xcode.app/Contents/Developer/
jpsim commented 9 years ago

Sorry I'm not following. You're saying it works for you for both the linked (almost empty) project, it works for a big iOS project, but not for a different Cocoa Touch Framework?

Can you share that framework for me to test? Otherwise it's difficult for me to troubleshoot.

juangamnik commented 9 years ago

Big is a little bit overstated, but yes. I'll try to create a minimalistic project showing the issue, ok? Please, give me some time.

jpsim commented 9 years ago

Sounds great, thanks for your help!

juangamnik commented 9 years ago

Thank you for your help!

I get the above mentioned error, e.g., if I move away a file from a project. The project will not build then in Xcode, too, but jazzy "just" returns the JSON parse error.

I don't have a missing file in the productive project I use, but could not reproduce the issue in a fresh project without doing nasty things like moving files out of the project.

My own framework has a build problem, too. It builds all well if I select the project root during building (⇧⌘K, ⌘B). Otherwise (e.g. if some swift file is selected) I get a <unknown>:0: error: could not build Objective-C module 'MyCustomFrameworkName'. But the minimalistic cocoa touch framework, you may find here has this problem too but builds well with jazzy as soon as you delete the connection to SomeProgGS.swift in the project...

Perhaps this JSON parse error is connected to another Xcode-problem?

jpsim commented 9 years ago

It sounds like you can only make jazzy fail with a broken Xcode project, which isn't something we want to support anyway. Thanks for investigating!

juangamnik commented 9 years ago

On the one hand the project is not broken. It works all fine. Xcode (and the swift/objective-c compiler) just seems to have a problem with custom frameworks, that has to do with the currently selected file (which will be repaired in the future as I hope).

On the other hand, jazzy should not end up with a JSON-error in this case, shouldn't it? If Xcode has a problem, jazzy should report a meaningful error message saying, that the project a problem.

In summary, I am unsure what the error causes. Breaking the Xcode project shows the same symptom as my project, but this does not mean, that these problems have the same cause.

I sent you the project because it shows, that the build problem, that my project has seems not to be the cause of the problem, as the simple project shows the same symptom but jazzy runs all fine.

I don't think that this is the point to leave the investigation with a satisfied feeling, that Xcode is the problem (or the project itself).

juangamnik commented 9 years ago 
 xcodebuild -dry-run

In the project root (of the real project) returns a lot of output and then:

** BUILD SUCCEEDED **

So I do not think that "broken project" is the root cause. I just cannot reproduce it with another project (without giving you all my sources). If you know an obfuscater for swift ...

jpsim commented 9 years ago

I don't think that this is the point to leave the investigation with a satisfied feeling, that Xcode is the problem (or the project itself).

Until I can reproduce the issue, troubleshoot, and fix it, I'm afraid there isn't much more to do. It sounds like you have a project that causes problems, but I don't have access to that :cry:.

If you were looking for a way to contribute to jazzy, there's your shot!

You can re-open the issue (or create a new one) with a more appropriate issue name once you determine what's wrong.

juangamnik commented 9 years ago

I would do this, but jazzy 'needs' (don't get me wrong here please) to give me a more appropriate error message like which file is the problem, which class/function/operator/global variable, or even perhaps an text excerpt of the doc that causes the trouble. Right now I'm fishing in the dark because the project runs all fine and jazzy gives me a cryptic error message 😢. Anyways, I wish you nice free days!!! Thanks for this project. Project documentation is so important.

jpsim commented 9 years ago

I've just pushed #124, which will log which Swift files couldn't be parsed and (hopefully) avoid creating invalid JSON. Can you please try running that branch on your project, @juangamnik? Thanks!

juangamnik commented 9 years ago

I'm not that familiar with ruby. Currently I install jazzy via gem. Do you have an installation guide for installing it from scratch?

jpsim commented 9 years ago

@juangamnik Yup, right there in the README: https://github.com/realm/jazzy#development

juangamnik commented 9 years ago

@jpsim Thank you very much (merry christmas and everything)

I found the cause of the issue. It is an extension to the core swift struct Array that I annotated with documentation:

/// Extends array via a shuffled function which returns a copy
/// of the array in randomized order.
extension Array {
    func shuffled() -> [T] {
        var list = self
        for i in 0..<(list.count - 1) {
            let j = Int(arc4random_uniform(UInt32(list.count - i))) + i
            swap(&list[i], &list[j])
        }
        return list
    }
}

Please add as much as possible helpful error output to future versions (and the next release should certainly contain this addition). This small line (the first of the next snippet) helped me a lot:

Set.swift could not be parsed. Please open an issue at https://github.com/jpsim/sourcekitten/issues with the file contents.
building site
/usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse': 399: unexpected token at '] (JSON::ParserError)
'
    from /usr/local/Cellar/ruby/2.1.5/lib/ruby/2.1.0/json/common.rb:155:in `parse'
    from /Users/juangamnik/blabla/jazzy/lib/jazzy/sourcekitten.rb:175:in `parse'
    from /Users/juangamnik/blabla/jazzy/lib/jazzy/doc_builder.rb:100:in `build_docs_for_sourcekitten_output'
    from /Users/juangamnik/blabla/jazzy/lib/jazzy/doc_builder.rb:53:in `build'
    from /Users/juangamnik/blabla/jazzy/bin/jazzy:15:in `<main>'

I could point out the problem in seconds. But it was very hard to pinpoint without this information.

Thank you very much. Now my project documentation works like a charm.