Open Manu343726 opened 9 years ago
Hi!
First of all thanks for all the work that you invested. I'm happy that someone uses this library ;)
I'd be happy to add oo-cmake to bii. Currently I do not have the time to do so. I'll get back to you as soon as I can.
Hi I'm going to try doing what you described now. I think I want to deploy on changes to master and when I create tags. would I be correct in assuming that after deploy: ...
I would add a on: \n tags: true 'n branch: master
and this would update the biicode block correctly?
Also: I actually do not use the c++ project currently (maybe in future when I decide to generate cmake code would I start programming somehting in c++) to run oo-cmake all you need is cmake/* and resources/* am I correct in assuming so I would write the followng into the biicode.conf
[dependencies]
oo-cmake.cmake + cmake/* resources/*
so I created a pull request - would you mind checking it If its ok I'll merge into master
Hi!
I'm glad to see you tried biicode and integrated a block into your library!
About the projects using oo-cmake, I'm working in some internal projects involving complex cmake scripts, and I think your library may help us. I have no public examples using oo-cmake currently, but the rest of the biicode team is so expectant about your library. If oo-cmake integrates with biicode as expected (And our (mine and yours) blocks give the impression that will be true :) ) we will be using it as a main tool in all our cmake scripts.
Now your questions:
What about the expression parser? Is not under the src/
directory? Please note that I'm not completely sure how your library processes your custom expressions. I'm correct?
For example, I assumed that src/
was needed, that's why you should disable the generated target (The one generated from the JSON parser) build. In my block, I use a foreach()
to disable all executable targets, since the oo-cmake block should be a CMake only block:
foreach(executable ${BII_BLOCK_EXES})
set_target_properties(manu343726_oo-cmake_${executable} PROPERTIES EXCLUDE_FROM_ALL 1 EXCLUDE_FROM_DEFAULT_BUILD 1)
endforeach()
I'm currently working on adding Boost support for biicode, but I'm having some problems with large Boost compilation times. Calling Boost build from execute_process()
produces no output at all during compilation, which is not fair from the UX point of view (Note that Boost build may take minutes, even hours).
Do you have any feature that may help with this, or at least makes possible to print a fancy progress text while running another command in background?
You can check the experimental Boost here. Note the scripts are under development (i.e. soooooooo ugly and full of sparse variables), but when the thing will work I have planned to simplify it via your library, with command wrappers, maps, etc. Please don't share the project, is still an experiment, too early to announce Boost support :)
Again, thanks a lot for all your feedback. All the biicode team will stay tuned to oo-cmake (if Boost with oo-cmake works, there are some of my mates talking about Qt via a oo-cmake based hook and more "big libraries", OpenCV, WxWidgets, etc). I hope oo-cmake will become a first-class tool for any cmake script. I really like it.
The expression parser is written in pure cmake (lang2, parse_script function using the expr.json as input file) and thus only needs the parser is also used to parse json (json.json)
so the only folder needed is resources.
I have written some helpers for processes see https://github.com/toeb/oo-cmake#parallel-processes the functions described can be used to start and wait for a process and for example emit a status indicator to the console.
sadly it is not completely (but almost) platform independent as it uses either bash or powerscript to start the processes. (untested on mac) (It could be really made platform independent using boost.process and a custom application)
maybe it'll help you ;)
travis does not deploy to biicode
Thanks! I was aware of your processes functionality, but I though the output was stored to a variable as execute_process()
does. But if I'm allowed to do message()
in a subprocess using your process_start_script()
this will be perfect for me. Let's print a bee with oo-cmake while building Boost! cool.
I noticed your oo-cmake biicode block is broken (I'm trying to depend on it :)), since it has no sources inside. Are you maintaining block by hand instead of using travis?
As usually, thanks a lot for your feedback.
Yeah this issue should still be open - travis does not deploy to bii because Skipping deployment with the biicode provider because this branch is not permitted to deploy
I'm still trying to work that out.
process_start_script
starts a cmake and hide the output - however the process is started asynchronously - so you can print anything as long as the process is running. also you can repeatedly read the stdout of the script you started. (Look at the unit tests to find out how this works)
You asked me for help to update the bii block on travis builds.
I think the best way is to use raw bii calls instead of the travis deploy service (It's a bit limited). Something like this:
language: cpp
os:
- linux
- osx
compiler:
- gcc
matrix:
allow_failures:
- os: osx
env:
global:
# github api token and secret
secure: "fWQnD0UqfX9SfctAu7lAnUJ4V5brp8UqdmPxQ67MlbmR5GAXQK9yezeQEsOnDCbihM6H/WTkonR920+B14fOb1NO0RSY39yHzjZG6B4tM6ZJwRW0CAlgFzlCqdwyL//7EhfKnxmWqflsA8oY+ihuZsj8P9ZtQSpNWENlu0Cl324="
secure: "i9rwJYZa6VDCpjXFaJoUggdn12mC1WB2zb9zaFP57VocTyL37Kpt3btXb9De5VQEJbyVKG2b9zlts20dEqvvruXBBuhWi+15VICT1yqq3zDQt25EPPDXw+AcB58dG+0icY1VKklWXR2Qxms4x/EZSN4TDNUp6LZgh+ACWlE1SRU="
matrix:
- CMAKE_DIR="v3.1" CMAKE_VERSION="3.1.2"
- CMAKE_DIR="v3.0" CMAKE_VERSION="3.0.1"
- CMAKE_DIR="v2.8" CMAKE_VERSION="2.8.12.2"
- CMAKE_DIR="v2.8" CMAKE_VERSION="2.8.7"
matrix:
allow_failures:
- env: CMAKE_DIR="v2.8" CMAKE_VERSION="2.8.7"
notifications:
slack: fallto:MlnVOMNkx8YopsaSSxqh2Rcr
before_install:
- sudo add-apt-repository --yes ppa:kalakris/cmake
- sudo apt-get update -qq
- sudo apt-get install mercurial
- sudo apt-get install git
- sudo apt-get install subversion
- git config --global user.email "travis-ci@example.com"
- git config --global user.name "Build Server"
install:
- wget http://apt.biicode.com/install.sh && chmod +x install.sh && ./install.sh
- bii setup:cpp
- bii init
- bii open toeb/cmakepp
- rsync -av --exclude="blocks" --exclude="bii" --exclude=".git" --exclude=".gitignore" . blocks/toeb/cmakepp > /dev/null
#- sudo apt-get install cmake
#- cmake --version
# install cmake --
- wget "http://www.cmake.org/files/${CMAKE_DIR}/cmake-${CMAKE_VERSION}.tar.gz"
- tar xf "cmake-${CMAKE_VERSION}.tar.gz"
- cmake -Hcmake-${CMAKE_VERSION} -B_builds -DCMAKE_BUILD_TYPE=Release -DCMAKE_USE_OPENSSL=ON -DCMAKE_INSTALL_PREFIX="`pwd`/_install"
- cmake --build _builds --target install
- export PATH="`pwd`/_install/bin:${PATH}"
# -- end
# verify installed cmake --
- which cmake
- cmake --version
#-- end
script:
- cmake -P build/script.cmake
after_success:
- "cmake -P build/after_success.cmake"
- bii user toeb -p $YOUR_ENCRYPTED_BII_PASSWORD
- bii publish toeb/cmakepp --tag STABLE
after_failure: "cmake -P build/after_failure.cmake"
after_script: "cmake -P build/after_script.cmake"
deploy:
- provider: releases
api_key:
secure: Y8fO6kLjCkPHi7qa1tkv9kVKvoEe/VsRO9gY8LFFsBliKFD1DC/+Sf1l5ij3SGawFduzCWWk4O62lEVawEnUhOqHFb+D6+SExVzF5QvkgSvnp6BkpyosvCn0/FXBy4tt4KZyLSJ3CzqLvDnD99d7vouHyxiLZy4JSTjmlnklkAo=
file: release/cmakepp.cmake
skip_cleanup: true
on:
tags: true
all_branches: true
Contributing on that script means I need your credentials to test publication. That's why I haven't sent you a pull request instead :(
Note how the block is published as STABLE
. This allows anyone to use your block. By default blocks published as DEV
cannot be reused by others except explicitly asked in user's policies.bii
file.
You can also use travis branch variables to set up different biicode tracks for your block matching each development branch you have on the repo. Check the python scripts I have at boost-biicode to generate different block tracks from block templates. These scripts for easy block generation and publication will be released anytime soon as an independent project
so if I understand correclty: I need to create an encrypted environment variable called YOUR_ENCRYPTED_BIIPASSWORD which contains my password and then the script you posted will work .?
I must confess I haven't used biicode much (as I am focussing so much on cmakepp) so terminology like tracks escapes me currently. Are they the biicode equivalent of tags/branches?
The only difference between your current travis script and the one I posted above is the publication method. In yours the travis deployment service is used, what I propose is to use manual publish (Note the bii publish
call on after_success
) instead. Using bii directly allows you to specify the kind of publication: DEV
(development, not intended to be used by users. It's the default mode). BETA
, ALPHA
, STABLE
(It's a stable release, users are able to use the block without explicit configuration).
I think the best way to handle bii publication is to release as STABLE
only on releases (That is, on git tagged commits). You can configure the travis script to run bii publish toeb/cmakepp --tag STABLE
only on releases if you like. I didn't suggest that first since, if I got your dev workflow correctly, you only merge/push to master if the new features were ready and tested in depth.
Note bii publish
command accepts an extra parameter to specify release version if you like.
This is the help from bii publish
:
usage: bii publish [-h] [--tag TAG] [--versiontag VERSIONTAG] [--msg MSG]
[--all] [-r [REMOTE]]
[block]
Publish (publicly) one or all the blocks of the current project.
positional arguments:
block Block name, e.g.: bii publish my_user/my_block. Do not
use with --all argument
optional arguments:
-h, --help show this help message and exit
--tag TAG Release life-cycle TAG, e.g: bii publish --tag ALPHA
--versiontag VERSIONTAG
Name tag for the version. e.g: v1.2 or "Sweet Beacon"
--msg MSG DEPRECATED: Publication description
--all Publish all blocks. Do not use with block argument
-r [REMOTE], --remote [REMOTE]
Publish VCS remote info. Format: "remote_url (branch)
@commit_id #tag" or blank to autodetect (currently
only git supported)
About tracks, you are correct, block tracks are our "branches" equivalent. The point is that you can maintain different tracks of a block, and later when you use the block you can specify what track you want via your dependencies settings (biicode.conf
file, [requirements]
entry).
As an example, we maintain an libuv block. There are three different tracks for that block representing the three different libuv versions we support. Another example is how I integrated cmakepp with boost-biicode hooks. What I did is to open your block, "fork it" i.e. create a new track from your block, and add the changes I need on my track of cmakepp. There's more information about tracks in the docs.
What I suggest is to match your dev workflow (becker + master branches) on biicode too, releasing in two correspondent tracks. This is only an idea, note is not necessary at all.
Don't worry about biicode usage, you are giving us a lot of useful feedback. Keep focused on your project, I know what maintaining a personal project on freetime means :P When cmakepp main features will be done, get a chance to test your lib with bii. You only have to include(toeb/cmakepp/cmakepp)
in any cmake file of a block :)
Hi! I'm Manu Sánchez, from the biicode team.
I'm working intensively with CMake scripts and I find your library so useful to me. Maps, function calls, etc; features that make feasible writing complex CMake scripts without going crazy!
biicode is a file-based dependency manager for C and C++, in the same spirit of Java's Maven and Maven Central or Python's pip. The tool uses CMake under the hood to process builds, so to setup a biicode block* you should play a bit with
CMakeLists.txt
files. A block is our unit of code sharing, think of it as a package.Here's an example, part of the
CMakeLists.txt
file of our Box2D block:It's exactly the same
CMakeLists.txt
from the original library, just adding a couple of toggles to make it work with biicode.To use Box2D, you only have to
#include
it in your C/C++ code, biicode does everything else (Download, setup build, etc). Check our docs for more info.Since version 2.0 we support dependency management of CMake scripts, via
include()
directives. Your library was very useful to me, so I created a block to deploy and reuse it easily across all my biicode blocks. Now I'm able to use your library inside my biicodeCMakeLists.txt
files just doinginclude(manu343726/oo-cmake/oo-cmake)
.I hope you like it.
oo-cmake changes
I tried not to touch your codebase at all, but use Travis CI to build and publish the block from your sources. So the only changes I propose you should be on the
.travis.yml
file (Among other minor changes on the README).The block
The idea is to have a biicode block with all your sources, while users access functionality via
include(manu343726/oo-cmake/oo-cmake)
. Check the block here.Dependency management
biicode is a file-based dependency management system. That means biicode uses files as its unit of dependency. So if you use
foo.cpp
from amanu343726/lib
block, biicode will download thefoo.cpp
file only, not the entiremanu343726/lib
block. This is interesting for large libraries and codebases, because only the#include
d files will be downloaded.But that does not work in this case. Your
oo-cmake.cmake
file depends in many CMake scripts and some extra C++ files that should be retrieved too, but you are not using ourinclude(user/block/script)
convention (And I'm not going to change your sources), so this dependency retrieving does not work directly.However, biicode allows you to specify dependencies manually in the
biicode.conf
file, the configuration file of the block:In our case, I specified explicitly that
oo-cmake.cmake
depends on the contents of thebuild/
,cmake/
,resources/
, andsrc/
folders. Now biicode will download that folders when theoo-cmake.cmake
file is used.CMakeLists.txt
Each biicode block has it's own
CMakeLists.txt
with building configuration. If no CMakeLists.txt is provided, biicode generates one. By default biicode searches for any C/C++ file adding it to block targets (Library or executable, depending on the kind of file: Is it a header?, is it a.cpp
with amain()
function?, etc).This process, done by the
ADD_BIICODE_TARGETS()
macro, works like a charm with C/C++ libraries, but not with CMake libraries that use some C/C++ files like yours. We don't want any C/C++ target in the oo-cmake block, we don't even want a target at all since this is designed to beinclude()
d only.The generated target is called
manu343726_oo-cmake_src_cmakeJsonParser_cmakeJsonParser
. SInce we don't want that target to be builded as part of our block, disable it after theADD_BIICODE_TARGETS()
call:Generating and publishing the block automatically
So we have to get the block, copy your library sources on it, write the custom
biicode.conf
, and edit the default biicodeCMakeLists.txt
to disable the undesired target. Then publish the block.If you suppose the block already exists this is easier, we just need to open the block and update its contents:
What I did is to create the block using the web inteface, add the custom
biicode.conf
andCMakeLists.txt
to it, and working in travis starting with that:Note that Travis CI supports biicode deployment out of the box, just install the travis gem and run biicode setup inside your project:
This command will ask you for biicode credentials, some configuration flags, and then will add the
deploy
entry to yourtravis.yml
.Accepting changes
This changes cannot be accepted directly via a pull request, because some changes depend on configuration based on my project:
manu343726
account, and the block ismanu343726/oo-cmake
. If you accept this proposal I suppose that you want your own biicode account and to release the oo-cmake with it. That said, I have no problem with owning the block, but this is not possible if you merge the proposal into your project (See next point).As I said previously, I tried not to modify your sources. Actually, this is the diff between my fork and your
master
branch:There are changes on the
README.md
too, the travis badge points to my builds, there's a cool biicode badge I generated with shields.io, etc.Since you have to perform some steps to accept this changes, not only to accept
.travis.yml
changes, I decided to put this as an issue with everything explained in depth instead of a pull request.Let's see how you can setup your project to deploy a biicode block automatically:
How to create your oo-cmake block
You can download our client from here.
Set up a new biicode project localy. This will be used to create and set up the block:
A folder
oo-cmake-setup
is created, with three directories inside:bii/
: biicode project metadata. Our.git/
counterpart.blocks
: Folder containing blocks being edited in the project. Each block has the pathblocks/username/blockname
.deps/
: Location of dependency blocks. Follows the same path convention as project blocks:deps/user/blockname
.Say we have a block
a
created with the accountuser
. That block has some sources depending on a blockb
created bydeveloper
. After runningbii find
(The command which searches and solves dependencies) our project looks like this:In this example,
main.cpp
froma
has an#include <developer/b/foo.hpp>
directive.See
getting started
from our docs for more info about project layout.Create the
oo-cmake
block:Create the
biicode.conf
file for the block, adding the explicit dependencies inside the[dependencies]
entry:**Create a
CMakeLists.txt
for the block with the target disable:Publish the block:
This will ask you for your account and password. Then the block is in the biicode cloud and we are ready to update it automatically via Travis CI builds.
Updating the block with Travis CI
Take your
.travis.yml
and add the following entries:Now run
travis setup biicode
. It will add the deploy entry to the.travis.yml
file:Note the
skip_cleanup
entry, it's very important! Since the block is opened and updated during build, we should specify Travis CI to not discard build changes before deploy. It should be added explicitly.That's all!
Feel free to ask me whatever question you want to, use the comments system bellow if you like.