fixtures.zip fails to make properly

[dormando]

unzip fixtures.zip
Archive:  fixtures.zip
warning [fixtures.zip]:  409970 extra bytes at beginning or within zipfile
  (attempting to process anyway)
[...]
  inflating: __MACOSX/fixtures/models/._cornellbox  
Makefile:16: recipe for target 'fixtures' failed
make: *** [fixtures] Error 1

Assuming the "extra bytes" warning causes zip to exit non-zero, which ends up leaving the __MACOSX and fixtures.zip garbage in the working directory.

on an ubuntu 16.04 machine.

[hunterloftis]

I wonder if this is a bsd/gnu incompatibility (I've only tested make fixtures on osx).

Thoughts on a better way to host > 1 GB of 3d asset data? I'd love something less hacky than this google drive wget, but don't have good ideas.

[dormando]

well wget is wget. probably zip behavior.

I ran it again:

$ wget --no-check-certificate -r 'https://docs.google.com/uc?export=download&id=1mNizqg4uSHbCILBwf1-fbrJDu6dtvA7R' -O fixtures.zip
WARNING: combining -O with -r or -p will mean that all downloaded content
will be placed in the single file you specified.

then it opens and downloads a bunch of html before finding the file.. which it appends and just happens to have the zip be the last thing it downloads. zip then makes a heroic attempt to decompress the file which has a bunch of HTML and a file listing smashed into it. sorry, I didn't actually look at what wget was doing the first time. Giving it recursive mode to find the dl link is clever in a gross way :)

I haven't looked at free hosting for large files. I use VM's or S3 buckets or etc. Looks like google drive is "unlimited, but maybe also limits".

TBH I would do one of two things:

1) have people explicitly go fetch the file via url. 2) include a util that uses the google drive API (or some other host API) and split the files up so "make lion" would download just that fixture if it wasn't already downloaded. I doubt most casual users are going to try every fixture in here and it's 1.8G uncompressed which is kind of a lot.

or maybe 3) tiny go thing that fetches fetches the URL to get the confirm nonce and directly downloads the zip :)

a bit of work to do that I guess?

[ghost]

How about a golang based downloader ? So people on different OS's dont get any issues and we have full control.

[ghost]

ah i just noticed its google drive and so in the browser it prompts for user interaction to download. thats a problem to automate

[dormando]

it's not too bad, the confirm link is in the HTML it gives you, but it's fragile since any change by google drive could break the tool. there're also copyright/attribution issues.

[ghost]

yeah not a show stopper. I just ran it and works fine.

Now i hit the issue of it not finding the fixture in the redblue demo.

go run redblue.go

Error: unable to open scene ./fixtures/models/mario/mario-sculpture.obj, open ./fixtures/models/mario/mario-sculpture.obj: no such file or directory

[ghost]

this fixed it for me:

    relPath := "../../fixtures/models/mario/mario-sculpture.obj"
    absPath, err := filepath.Abs(relPath)

    mesh, err := obj.ReadFile(absPath, true)
    if err != nil {
        return err
    }

[ghost]

btw here is a golag downloader that supports retry and is stateless. I could not get it working with google drive but it will work wit google storage and s3 if you even change.

package main

import (
    "fmt"
    "os"
    "time"

    "github.com/cavaliercoder/grab"
)

func main() {
    // create client
    client := grab.NewClient()

    // url := http://www.golang-book.com/public/pdf/gobook.pdf"
    url := "https://docs.google.com/uc?export=download&id=1mNizqg4uSHbCILBwf1-fbrJDu6dtvA7R?acknowledgeAbuse=true"
    req, _ := grab.NewRequest(".", url)

    // start download
    fmt.Printf("Downloading %v...\n", req.URL())
    resp := client.Do(req)
    fmt.Printf("  %v\n", resp.HTTPResponse.Status)

    // start UI loop
    t := time.NewTicker(500 * time.Millisecond)
    defer t.Stop()

Loop:
    for {
        select {
        case <-t.C:
            fmt.Printf("  transferred %v / %v bytes (%.2f%%)\n",
                resp.BytesComplete(),
                resp.Size,
                100*resp.Progress())

        case <-resp.Done:
            // download is complete
            break Loop
        }
    }

    // check for errors
    if err := resp.Err(); err != nil {
        fmt.Fprintf(os.Stderr, "Download failed: %v\n", err)
        os.Exit(1)
    }

    fmt.Printf("Download saved to ./%v \n", resp.Filename)

}

[hunterloftis]

@gedw99 the examples are designed to be run from the package root dir (where Makefile is):

$ go run examples/redblue/redblue.go

[ghost]

Ah makes sense.

On Sun, 2 Sep 2018, 02:41 Hunter Loftis, notifications@github.com wrote:

@gedw99 https://github.com/gedw99 the examples are designed to be run from the package root dir (where Makefile is):

$ go run examples/redblue/redblue.go

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hunterloftis/pbr/issues/37#issuecomment-417896938, or mute the thread https://github.com/notifications/unsubscribe-auth/ATuCwtQKAysrb_mcyNkaFy5GKL2hweInks5uWylWgaJpZM4WUJds .