hooram / ownphotos

Self hosted alternative to Google Photos
MIT License
2.77k stars 233 forks source link

docker image for arm distributions #55

Open jgonzale opened 5 years ago

jgonzale commented 5 years ago

Hi there!

I'm so interesting on experimenting with ownphotos in my modest NAS server, an Odroid HC2. Unfortunately, I'm not able to run it through docker so I presume it is build for x86/x86_64 system and the Odroid uses ARMv7-A processors.

Would it be possible somehow to port the docker image to this architecture? Otherwise, could you point the files I should modify so as to try myself the port? I'm not very familiar with docker configurations more than running or tuning images, but I could give it a try.

Many thanks!!

hooram commented 5 years ago

From a quick look at pytorch and face_recognition pages, it seems that both should be able to run on ARM processors. I would try and follow the installation steps in Dockerfile, and try to get all the dependencies to install on an ARM processor. I think that most of the problems would be met with the two packages I mentioned, pytorch and face_recognition and their dependencies.

I think a bigger issue is that it might poop out (memory restriction, etc) in run time than getting it to install on a low resource device. Hope this helps.

guysoft commented 5 years ago

The main challenge on my side is that I can't see how to make automated build for arm on docker hub. It might require building an automation and pushing it to dockerhub. If anyone has ever build any raspberrypi docker image it would be great if you point me how to do that.

It should be doable, even if pytorch does not work (and it should).

guysoft commented 5 years ago

Ok, should be something like this: https://forums.docker.com/t/automated-build-raspberry-pi-based-image-on-docker-hub/4155/7

I can try and run a build locally here and push it manually, then we can see if it works, if it does we can automate it.

guysoft commented 5 years ago

There seems to be only armv7 image available: https://hub.docker.com/r/arm32v7/debian

This means no support for armv6 raspebrrypi1 and raspberypi zero, but those are super slow anyway so its a releasable requirement.

This is the debian docker image: https://hub.docker.com/r/arm32v7/debian

I am running a build on a raspberrypi and see if that works. In the longer run using something to run nighly builds using qemu-arm-static similar to I do in CustomPiOS would be a better option.

guysoft commented 5 years ago

Ok, seems like dilib build fails on arm, might be just lack of RAM:

It is doable and there are instructions here: https://www.pyimagesearch.com/2017/05/01/install-dlib-raspberry-pi/ Trying to build without it to see if that narrows down the build issues.

[ 85%] Building CXX object CMakeFiles/dlib_python.dir/src/vector.cpp.o
In file included from /usr/include/c++/6/vector:69:0,
                 from /dlib/dlib/external/pybind11/include/pybind11/detail/common.h:140,
                 from /dlib/dlib/external/pybind11/include/pybind11/pytypes.h:12,
                 from /dlib/dlib/external/pybind11/include/pybind11/cast.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/attr.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/pybind11.h:43,
                 from /dlib/dlib/../dlib/python/pybind_utils.h:6,
                 from /dlib/dlib/../dlib/python.h:6,
                 from /dlib/tools/python/src/opaque_types.h:6,
                 from /dlib/tools/python/src/vector.cpp:4:
/usr/include/c++/6/bits/vector.tcc: In member function 'std::vector<_Tp, _Alloc>::iterator std::vector<_Tp, _Alloc>::_M_erase(std::vector<_Tp, _Alloc>::iterator) [with _Tp = dlib::vector<double, 2l>; _Alloc = std::allocator<dlib::vector<double, 2l> >]':
/usr/include/c++/6/bits/vector.tcc:141:5: note: parameter passing for argument of type 'std::vector<dlib::vector<double, 2l> >::iterator {aka __gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >}' will change in GCC 7.1
     vector<_Tp, _Alloc>::
     ^~~~~~~~~~~~~~~~~~~
/usr/include/c++/6/bits/vector.tcc: In member function 'void std::vector<_Tp, _Alloc>::_M_insert_aux(std::vector<_Tp, _Alloc>::iterator, _Args&& ...) [with _Args = {const dlib::vector<double, 2l>&}; _Tp = dlib::vector<double, 2l>; _Alloc = std::allocator<dlib::vector<double, 2l> >]':
/usr/include/c++/6/bits/vector.tcc:321:7: note: parameter passing for argument of type 'std::vector<dlib::vector<double, 2l> >::iterator {aka __gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >}' will change in GCC 7.1
       vector<_Tp, _Alloc>::
       ^~~~~~~~~~~~~~~~~~~
/usr/include/c++/6/bits/vector.tcc:321:7: note: parameter passing for argument of type 'std::vector<dlib::vector<double, 2l> >::iterator {aka __gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >}' will change in GCC 7.1
In file included from /usr/include/c++/6/algorithm:62:0,
                 from /dlib/dlib/../dlib/algs.h:106,
                 from /dlib/dlib/../dlib/serialize.h:142,
                 from /dlib/dlib/../dlib/python/pybind_utils.h:9,
                 from /dlib/dlib/../dlib/python.h:6,
                 from /dlib/tools/python/src/opaque_types.h:6,
                 from /dlib/tools/python/src/vector.cpp:4:
/usr/include/c++/6/bits/stl_algo.h: In function '_RandomAccessIterator std::__find_if(_RandomAccessIterator, _RandomAccessIterator, _Predicate, std::random_access_iterator_tag) [with _RandomAccessIterator = __gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >; _Predicate = __gnu_cxx::__ops::_Iter_equals_val<const dlib::vector<double, 2l> >]':
/usr/include/c++/6/bits/stl_algo.h:112:5: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
     __find_if(_RandomAccessIterator __first, _RandomAccessIterator __last,
     ^~~~~~~~~
/usr/include/c++/6/bits/stl_algo.h:112:5: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
/usr/include/c++/6/bits/stl_algo.h: In function '_RandomAccessIterator std::__find_if(_RandomAccessIterator, _RandomAccessIterator, _Predicate, std::random_access_iterator_tag) [with _RandomAccessIterator = __gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >; _Predicate = __gnu_cxx::__ops::_Iter_equals_val<const dlib::vector<double, 2l> >]':
/usr/include/c++/6/bits/stl_algo.h:112:5: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
/usr/include/c++/6/bits/stl_algo.h:112:5: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
In file included from /usr/include/c++/6/vector:69:0,
                 from /dlib/dlib/external/pybind11/include/pybind11/detail/common.h:140,
                 from /dlib/dlib/external/pybind11/include/pybind11/pytypes.h:12,
                 from /dlib/dlib/external/pybind11/include/pybind11/cast.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/attr.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/pybind11.h:43,
                 from /dlib/dlib/../dlib/python/pybind_utils.h:6,
                 from /dlib/dlib/../dlib/python.h:6,
                 from /dlib/tools/python/src/opaque_types.h:6,
                 from /dlib/tools/python/src/vector.cpp:4:
/usr/include/c++/6/bits/vector.tcc: In member function 'void std::vector<_Tp, _Alloc>::_M_range_insert(std::vector<_Tp, _Alloc>::iterator, _ForwardIterator, _ForwardIterator, std::forward_iterator_tag) [with _ForwardIterator = __gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >; _Tp = dlib::vector<double, 2l>; _Alloc = std::allocator<dlib::vector<double, 2l> >]':
/usr/include/c++/6/bits/vector.tcc:616:7: note: parameter passing for argument of type 'std::vector<dlib::vector<double, 2l> >::iterator {aka __gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >}' will change in GCC 7.1
       vector<_Tp, _Alloc>::
       ^~~~~~~~~~~~~~~~~~~
/usr/include/c++/6/bits/vector.tcc:616:7: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
/usr/include/c++/6/bits/vector.tcc:616:7: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
In file included from /usr/include/c++/6/algorithm:62:0,
                 from /dlib/dlib/../dlib/algs.h:106,
                 from /dlib/dlib/../dlib/serialize.h:142,
                 from /dlib/dlib/../dlib/python/pybind_utils.h:9,
                 from /dlib/dlib/../dlib/python.h:6,
                 from /dlib/tools/python/src/opaque_types.h:6,
                 from /dlib/tools/python/src/vector.cpp:4:
/usr/include/c++/6/bits/stl_algo.h: In lambda function:
/usr/include/c++/6/bits/stl_algo.h:162:43: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
          std::__iterator_category(__first));
                                           ^
In file included from /usr/include/c++/6/vector:64:0,
                 from /dlib/dlib/external/pybind11/include/pybind11/detail/common.h:140,
                 from /dlib/dlib/external/pybind11/include/pybind11/pytypes.h:12,
                 from /dlib/dlib/external/pybind11/include/pybind11/cast.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/attr.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/pybind11.h:43,
                 from /dlib/dlib/../dlib/python/pybind_utils.h:6,
                 from /dlib/dlib/../dlib/python.h:6,
                 from /dlib/tools/python/src/opaque_types.h:6,
                 from /dlib/tools/python/src/vector.cpp:4:
/usr/include/c++/6/bits/stl_vector.h: In lambda function:
/usr/include/c++/6/bits/stl_vector.h:1148:58: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
       { return _M_erase(begin() + (__position - cbegin())); }
                                                          ^
In file included from /usr/include/c++/6/algorithm:62:0,
                 from /dlib/dlib/../dlib/algs.h:106,
                 from /dlib/dlib/../dlib/serialize.h:142,
                 from /dlib/dlib/../dlib/python/pybind_utils.h:9,
                 from /dlib/dlib/../dlib/python.h:6,
                 from /dlib/tools/python/src/opaque_types.h:6,
                 from /dlib/tools/python/src/vector.cpp:4:
/usr/include/c++/6/bits/stl_algo.h:162:43: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
          std::__iterator_category(__first));
                                           ^
In file included from /usr/include/c++/6/vector:64:0,
                 from /dlib/dlib/external/pybind11/include/pybind11/detail/common.h:140,
                 from /dlib/dlib/external/pybind11/include/pybind11/pytypes.h:12,
                 from /dlib/dlib/external/pybind11/include/pybind11/cast.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/attr.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/pybind11.h:43,
                 from /dlib/dlib/../dlib/python/pybind_utils.h:6,
                 from /dlib/dlib/../dlib/python.h:6,
                 from /dlib/tools/python/src/opaque_types.h:6,
                 from /dlib/tools/python/src/vector.cpp:4:
/usr/include/c++/6/bits/stl_vector.h: In lambda function:
/usr/include/c++/6/bits/stl_vector.h:1375:4: note: parameter passing for argument of type 'std::vector<dlib::vector<double, 2l> >::iterator {aka __gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >}' will change in GCC 7.1
    _M_range_insert(__pos, __first, __last, _IterCategory());
    ^~~~~~~~~~~~~~~
/usr/include/c++/6/bits/stl_vector.h:1375:4: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
/usr/include/c++/6/bits/stl_vector.h:1375:4: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
/usr/include/c++/6/bits/stl_vector.h:1375:4: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
In file included from /usr/include/c++/6/algorithm:62:0,
                 from /dlib/dlib/../dlib/algs.h:106,
                 from /dlib/dlib/../dlib/serialize.h:142,
                 from /dlib/dlib/../dlib/python/pybind_utils.h:9,
                 from /dlib/dlib/../dlib/python.h:6,
                 from /dlib/tools/python/src/opaque_types.h:6,
                 from /dlib/tools/python/src/vector.cpp:4:
/usr/include/c++/6/bits/stl_algo.h: In static member function 'static pybind11::handle pybind11::cpp_function::initialize(Func&&, Return (*)(Args ...), const Extra& ...) [with Func = pybind11::detail::vector_if_equal_operator(pybind11::detail::enable_if_t<pybind11::detail::is_comparable<Vector>::value, Class_>&) [with Vector = std::vector<dlib::vector<double, 2l> >; Class_ = pybind11::class_<std::vector<dlib::vector<double, 2l> >, std::unique_ptr<std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > >, std::default_delete<std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > > > > >]::<lambda(std::vector<dlib::vector<double, 2l> >&, const T&)>; Return = void; Args = {std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > >&, const dlib::vector<double, 2l>&}; Extra = {pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, char [95]}]::<lambda(pybind11::detail::function_call&)>::_FUN(pybind11::detail::function_call&)':
/usr/include/c++/6/bits/stl_algo.h:162:43: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
          std::__iterator_category(__first));
                                           ^
In file included from /usr/include/c++/6/vector:64:0,
                 from /dlib/dlib/external/pybind11/include/pybind11/detail/common.h:140,
                 from /dlib/dlib/external/pybind11/include/pybind11/pytypes.h:12,
                 from /dlib/dlib/external/pybind11/include/pybind11/cast.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/attr.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/pybind11.h:43,
                 from /dlib/dlib/../dlib/python/pybind_utils.h:6,
                 from /dlib/dlib/../dlib/python.h:6,
                 from /dlib/tools/python/src/opaque_types.h:6,
                 from /dlib/tools/python/src/vector.cpp:4:
/usr/include/c++/6/bits/stl_vector.h: In static member function 'static pybind11::handle pybind11::cpp_function::initialize(Func&&, Return (*)(Args ...), const Extra& ...) [with Func = pybind11::detail::vector_modifiers(pybind11::detail::enable_if_t<pybind11::detail::is_copy_constructible<typename Container::value_type>::value, Class_>&) [with Vector = std::vector<dlib::vector<double, 2l> >; Class_ = pybind11::class_<std::vector<dlib::vector<double, 2l> >, std::unique_ptr<std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > >, std::default_delete<std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > > > > >]::<lambda(std::vector<dlib::vector<double, 2l> >&, const std::vector<dlib::vector<double, 2l> >&)>; Return = void; Args = {std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > >&, const std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > >&}; Extra = {pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, char [61]}]::<lambda(pybind11::detail::function_call&)>::_FUN(pybind11::detail::function_call&)':
/usr/include/c++/6/bits/stl_vector.h:1375:4: note: parameter passing for argument of type 'std::vector<dlib::vector<double, 2l> >::iterator {aka __gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >}' will change in GCC 7.1
    _M_range_insert(__pos, __first, __last, _IterCategory());
    ^~~~~~~~~~~~~~~
/usr/include/c++/6/bits/stl_vector.h:1375:4: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
/usr/include/c++/6/bits/stl_vector.h:1375:4: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
/usr/include/c++/6/bits/stl_vector.h:1375:4: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
In file included from /usr/include/c++/6/algorithm:62:0,
                 from /dlib/dlib/../dlib/algs.h:106,
                 from /dlib/dlib/../dlib/serialize.h:142,
                 from /dlib/dlib/../dlib/python/pybind_utils.h:9,
                 from /dlib/dlib/../dlib/python.h:6,
                 from /dlib/tools/python/src/opaque_types.h:6,
                 from /dlib/tools/python/src/vector.cpp:4:
/usr/include/c++/6/bits/stl_algo.h: In static member function 'static pybind11::handle pybind11::cpp_function::initialize(Func&&, Return (*)(Args ...), const Extra& ...) [with Func = pybind11::detail::vector_if_equal_operator(pybind11::detail::enable_if_t<pybind11::detail::is_comparable<Vector>::value, Class_>&) [with Vector = std::vector<dlib::vector<double, 2l> >; Class_ = pybind11::class_<std::vector<dlib::vector<double, 2l> >, std::unique_ptr<std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > >, std::default_delete<std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > > > > >]::<lambda(const std::vector<dlib::vector<double, 2l> >&, const T&)>; Return = bool; Args = {const std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > >&, const dlib::vector<double, 2l>&}; Extra = {pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, char [41]}]::<lambda(pybind11::detail::function_call&)>::_FUN(pybind11::detail::function_call&)':
/usr/include/c++/6/bits/stl_algo.h:162:43: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<const dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
          std::__iterator_category(__first));
                                           ^
In file included from /usr/include/c++/6/vector:64:0,
                 from /dlib/dlib/external/pybind11/include/pybind11/detail/common.h:140,
                 from /dlib/dlib/external/pybind11/include/pybind11/pytypes.h:12,
                 from /dlib/dlib/external/pybind11/include/pybind11/cast.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/attr.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/pybind11.h:43,
                 from /dlib/dlib/../dlib/python/pybind_utils.h:6,
                 from /dlib/dlib/../dlib/python.h:6,
                 from /dlib/tools/python/src/opaque_types.h:6,
                 from /dlib/tools/python/src/vector.cpp:4:
/usr/include/c++/6/bits/stl_vector.h: In lambda function:
/usr/include/c++/6/bits/stl_vector.h:1148:58: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
       { return _M_erase(begin() + (__position - cbegin())); }
                                                          ^
In file included from /usr/include/c++/6/vector:69:0,
                 from /dlib/dlib/external/pybind11/include/pybind11/detail/common.h:140,
                 from /dlib/dlib/external/pybind11/include/pybind11/pytypes.h:12,
                 from /dlib/dlib/external/pybind11/include/pybind11/cast.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/attr.h:13,
                 from /dlib/dlib/external/pybind11/include/pybind11/pybind11.h:43,
                 from /dlib/dlib/../dlib/python/pybind_utils.h:6,
                 from /dlib/dlib/../dlib/python.h:6,
                 from /dlib/tools/python/src/opaque_types.h:6,
                 from /dlib/tools/python/src/vector.cpp:4:
/usr/include/c++/6/bits/vector.tcc: In static member function 'static pybind11::handle pybind11::cpp_function::initialize(Func&&, Return (*)(Args ...), const Extra& ...) [with Func = pybind11::detail::vector_modifiers(pybind11::detail::enable_if_t<pybind11::detail::is_copy_constructible<typename Container::value_type>::value, Class_>&) [with Vector = std::vector<dlib::vector<double, 2l> >; Class_ = pybind11::class_<std::vector<dlib::vector<double, 2l> >, std::unique_ptr<std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > >, std::default_delete<std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > > > > >]::<lambda(std::vector<dlib::vector<double, 2l> >&, SizeType, const T&)>; Return = void; Args = {std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > >&, unsigned int, const dlib::vector<double, 2l>&}; Extra = {pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg, char [36]}]::<lambda(pybind11::detail::function_call&)>::_FUN(pybind11::detail::function_call&)':
/usr/include/c++/6/bits/vector.tcc:128:8: note: parameter passing for argument of type 'const __gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
        _M_insert_aux(__pos, std::move(__x_copy));
        ^~~~~~~~~~~~~
/usr/include/c++/6/bits/vector.tcc:131:6: note: parameter passing for argument of type 'const __gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
      _M_insert_aux(__pos, __x);
      ^~~~~~~~~~~~~
/usr/include/c++/6/bits/vector.tcc: In lambda function:
/usr/include/c++/6/bits/vector.tcc:128:8: note: parameter passing for argument of type 'const __gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
        _M_insert_aux(__pos, std::move(__x_copy));
        ^~~~~~~~~~~~~
/usr/include/c++/6/bits/vector.tcc:131:6: note: parameter passing for argument of type 'const __gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
      _M_insert_aux(__pos, __x);
      ^~~~~~~~~~~~~
In file included from /dlib/dlib/../dlib/python/pybind_utils.h:6:0,
                 from /dlib/dlib/../dlib/python.h:6,
                 from /dlib/tools/python/src/opaque_types.h:6,
                 from /dlib/tools/python/src/vector.cpp:4:
/dlib/dlib/external/pybind11/include/pybind11/pybind11.h: In function 'pybind11::iterator pybind11::make_iterator(Iterator, Sentinel, Extra&& ...) [with pybind11::return_value_policy Policy = (pybind11::return_value_policy)6u; Iterator = __gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >; Sentinel = __gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >; ValueType = dlib::vector<double, 2l>&; Extra = {}]':
/dlib/dlib/external/pybind11/include/pybind11/pybind11.h:1526:10: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
 iterator make_iterator(Iterator first, Sentinel last, Extra &&... extra) {
          ^~~~~~~~~~~~~
/dlib/dlib/external/pybind11/include/pybind11/pybind11.h:1526:10: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
In file included from /dlib/tools/python/src/opaque_types.h:8:0,
                 from /dlib/tools/python/src/vector.cpp:4:
/dlib/dlib/external/pybind11/include/pybind11/stl_bind.h: In lambda function:
/dlib/dlib/external/pybind11/include/pybind11/stl_bind.h:274:12: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
            },
            ^
/dlib/dlib/external/pybind11/include/pybind11/stl_bind.h:274:12: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
/dlib/dlib/external/pybind11/include/pybind11/stl_bind.h:274:12: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
/dlib/dlib/external/pybind11/include/pybind11/stl_bind.h: In static member function 'static pybind11::handle pybind11::cpp_function::initialize(Func&&, Return (*)(Args ...), const Extra& ...) [with Func = pybind11::detail::vector_accessor(pybind11::detail::enable_if_t<(! pybind11::detail::negation<std::is_same<decltype (declval<Vector>()[typename Vector::size_type()]), typename Vector::value_type&> >::value), Class_>&) [with Vector = std::vector<dlib::vector<double, 2l> >; Class_ = pybind11::class_<std::vector<dlib::vector<double, 2l> >, std::unique_ptr<std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > >, std::default_delete<std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > > > > >]::<lambda(std::vector<dlib::vector<double, 2l> >&)>; Return = pybind11::iterator; Args = {std::vector<dlib::vector<double, 2l>, std::allocator<dlib::vector<double, 2l> > >&}; Extra = {pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::keep_alive<0u, 1u>}]::<lambda(pybind11::detail::function_call&)>::_FUN(pybind11::detail::function_call&)':
/dlib/dlib/external/pybind11/include/pybind11/stl_bind.h:274:12: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
            },
            ^
/dlib/dlib/external/pybind11/include/pybind11/stl_bind.h:274:12: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
/dlib/dlib/external/pybind11/include/pybind11/stl_bind.h:274:12: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<dlib::vector<double, 2l>*, std::vector<dlib::vector<double, 2l> > >' will change in GCC 7.1
[ 85%] Building CXX object CMakeFiles/dlib_python.dir/src/svm_c_trainer.cpp.o
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-6/README.Bugs> for instructions.
make[2]: *** [CMakeFiles/dlib_python.dir/src/svm_c_trainer.cpp.o] Error 4
make[1]: *** [CMakeFiles/dlib_python.dir/all] Error 2
make: *** [all] Error 2
CMakeFiles/dlib_python.dir/build.make:134: recipe for target 'CMakeFiles/dlib_python.dir/src/svm_c_trainer.cpp.o' failed
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/dlib_python.dir/all' failed
Makefile:83: recipe for target 'all' failed
Traceback (most recent call last):
  File "setup.py", line 258, in <module>
    'Topic :: Software Development',
  File "/venv/lib/python3.5/site-packages/setuptools/__init__.py", line 143, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/venv/lib/python3.5/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/venv/lib/python3.5/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/venv/lib/python3.5/site-packages/setuptools/command/bdist_egg.py", line 172, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/venv/lib/python3.5/site-packages/setuptools/command/bdist_egg.py", line 158, in call_command
    self.run_command(cmdname)
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/venv/lib/python3.5/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/usr/lib/python3.5/distutils/command/install_lib.py", line 109, in build
    self.run_command('build_ext')
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "setup.py", line 133, in run
    self.build_extension(ext)
  File "setup.py", line 173, in build_extension
    subprocess.check_call(cmake_build, cwd=build_folder)
  File "/usr/lib/python3.5/subprocess.py", line 271, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j1']' returned non-zero exit status 2
ERROR: Service 'backend' failed to build: The command '/bin/sh -c apt-get update &&     apt-get install -y cmake git &&     git clone https://github.com/davisking/dlib.git &&     mkdir /dlib/build &&     cd /dlib/build &&     cmake .. -DDLIB_USE_CUDA=0 -DUSE_AVX_INSTRUCTIONS=0 &&     cmake --build . &&     cd /dlib &&     /venv/bin/python setup.py install --no USE_AVX_INSTRUCTIONS --no DLIB_USE_CUDA' returned a non-zero code: 1
pi@fullpageos:~/ownphotos $  
guysoft commented 5 years ago

pytorch also is not supported, it would need to be built from source. Unless torchvision package is good enough (its what pytorch put on their site now). @hooram Where did you see it could run on arm?

hooram commented 5 years ago

I initially saw it on https://pytorch.org/blog/the-road-to-1_0/, but it seems that I misread the information (toward the bottom of the intro section). It doesn't explicitly say that pytorch supports ARM out of the box, but rather

At Facebook (the largest stakeholder for PyTorch) we have Caffe2, which has been the production-ready platform, running in our datacenters and shipping to more than 1 billion phones spanning eight generations of iPhones and six generations of Android CPU architectures. It has server-optimized inference on Intel / ARM, TensorRT support, and all the necessary bits for production.

Either way, I found some guy who successfully installed pytorch v0.3.1 on a Raspberry Pi 3 here. I haven't found any information on installing the version of pytorch Ownphotos uses currently, which is pytorch v0.4.0. Actually, Ownphotos was using pytorch v0.3.1 before ec984dd1ff5eba0a0163b8dae50ea863e110518a.

So technically we should be able to get it to work on ARM processors, but regretfully I don't think I'll be able to work on that. I really appreciate you all looking into the matter nonetheless.

guysoft commented 5 years ago

@hooram it can be built, but it needs special treatment not to run out of memory: https://github.com/pytorch/pytorch/issues/7841#issuecomment-392105536

guysoft commented 5 years ago

Found that brewblox cross-compille base images, this could be used in ownphtos:

https://brewblox.netlify.com/dev/decisions/crosscompilation_base_images.html#context

https://github.com/BrewBlox/brewblox-deployment

guysoft commented 5 years ago

Hey, Started work on this, got as far as letting docker build the whole weekend pytorch (yes builds take days in the context), and a few other deps. The issues left that are known are:

  1. faiss needs to be built from source
  2. The env uses now miniconda which does not come for arm, so I need to get it working on pip3 environment. Should not be too hard since it worked on that before, and the reason we moved to miniconda is due to errors in pytorch.
  3. Few other python things.

If you want to aid in development the image is at: https://cloud.docker.com/repository/docker/guysoft/ownphotos-raspberrypi/

You can see sources at: https://github.com/guysoft/ownphotos/blob/raspberrypi/Dockerfile

Pytorch docker container at: https://github.com/guysoft/pytorch-docker-armv7

guysoft commented 5 years ago

Ok, I got as far as starting the backend on RaspberryPi, aaaand it runs out of RAM:

root@572fde4f7b05:/code# python3 manage.py startapp bur
/usr/local/lib/python3.6/dist-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
Killed

dmesg:

[ 3021.369927] Out of memory: Kill process 7422 (python3) score 592 or sacrifice child

I have 930MB free on my Pi.

So ODROID might be able to run the image cc @jgonzale. But I have no hardware to test.

edoaxyz commented 5 years ago

Try to set a larger swap memory

guysoft commented 5 years ago

Ok, increasing the swap using this guide to 2GB seems to get it to start the backend.

However going via the nginx guicron gives:

[2019-05-15 15:17:36 +0000] [401] [CRITICAL] WORKER TIMEOUT (pid:455)
[2019-05-15 15:17:36 +0000] [401] [CRITICAL] WORKER TIMEOUT (pid:456)
[2019-05-15 15:17:36 +0000] [455] [INFO] Worker exiting (pid: 455)
[2019-05-15 15:17:36 +0000] [456] [INFO] Worker exiting (pid: 456)
[2019-05-15 15:17:37 +0000] [467] [INFO] Booting worker with pid: 467
[2019-05-15 15:17:37 +0000] [468] [INFO] Booting worker with pid: 468
/usr/local/lib/python3.6/dist-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
/usr/local/lib/python3.6/dist-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
[2019-05-15 15:18:07 +0000] [401] [CRITICAL] WORKER TIMEOUT (pid:467)
[2019-05-15 15:18:07 +0000] [401] [CRITICAL] WORKER TIMEOUT (pid:468)
[2019-05-15 15:18:08 +0000] [467] [INFO] Worker exiting (pid: 467)
[2019-05-15 15:18:08 +0000] [468] [INFO] Worker exiting (pid: 468)
[2019-05-15 15:18:08 +0000] [479] [INFO] Booting worker with pid: 479
[2019-05-15 15:18:08 +0000] [480] [INFO] Booting worker with pid: 480

I managed to start it using:

python3 manage.py runserver

I am not sure how to solve this timeout.

gnn06 commented 4 years ago

Hi, Trying to install ownphoto on a raspberry pi 2 (32bit armv7). I use https://github.com/guysoft/ownphotos.git on branch raspberrypi. I get the "internal compiler error: Killed (program cc1plus)" error building the backend docker service. I solve by changing the swap file size from 100 Mo (defaut) to 1 Go. My raspberry pi 2+ has 1 Go RAM. Raspian buster. I continue to build the docker image. Thanks