Closed whedon closed 5 years ago
Friendly reminder that weβre waiting on your input here @boshek :smile:
@arfon. Sorry this got missed on the last reminder for some reason. I will allot some time towards this.
Hello @MarionLegrandLogrami and @arfon
It is possible that I am out of depth here. I acknowledge that. However if I am asked to provide an honest assessment here I don't think that the authors have addressed my concerns above. I am having trouble getting the database set up on Windows. My original recommendation is that all the database set up be moved into R functions; that is wrap the shell script in an R function. In the absence of functions that do this the installation instruction need to be comprehensive - that is step by step. I am making the assumption here that the audience of this package is an average R user without any (or maybe only a little) knowledge of PostgreSQL. With that in mind this section:
https://github.com/MarionLegrandLogrami/stacomiR/tree/master/Installation#22-on-windows-host-
assumes too much knowledge on the users part.
I am unable to call the install_stacomi.cmd
. Perhaps there is additional setup, which may be obvious, but if I am the naive new user I don't think that I would be the only person to run into these issues. For example this line:
You WILL have to give a SQL user with admin rights (create a database, create a connection role). Else the install will fail.
assumes that a user even knows what this means or how to go about doing it. My assessment here might be reflective of my relatively poor understanding of PostgreSQL. However if the authors have a different audience for the package in mind, rather than an R user with limited knowledge of databases, perhaps this is fine. I wish to be clear in saying that this is not an indictment of the authors work. The skills and technical expertise is clear and impressive. However I am still not able to get this up an running using the current instructions.
Thanks for your honest feedback here @boshek - it's much appreciated.
@MarionLegrandLogrami - both of the reviewers of this submission have had serious challenges getting the database up and running for testing. While @boshek is being very gracious stating that they don't feel like an expert with these technologies I think their point that this is going to be very hard for any users of your software is a very good one.
I'm not exactly sure what to suggest at this point but it's clear that the installation documentation is still lacking for new users. Perhaps you could find someone local to your host institution to try and follow the installation instructions so you can get a better understanding of what challenges people are running into.
As things stand, both reviewers have dedicated significant time and effort to these reviews but thus far, it's not been possible for either of our reviewers to fully verify the functionality of this software which means that this review is currently blocked.
Hi @boshek and @arfon,
The progam we provided to help to install everything is a cmd file. So it is an executable file. To launch it, you have to double-clik on the file. Doing this you will launch the program with some default settings. If you want to change these default settings you have to call the program in a console (start menu / cmd+enter to call the command) and enter the settings you need. I made some changes to the documentation in github to explain more clearly how to do this. If you really want to execute this program inside R, you can call the shell function. I explained everything in the documentation:
scriptAddr<-"path_to_install_stacomi.cmd" (e.g. "C:/Users/Public/stacomir/Installation/install_stacomi.cmd") shell(scriptAddr,"cmd")
scriptAddr<-"path_to_install_stacomi.cmd" (e.g. "C:/Users/Public/stacomir/Installation/install_stacomi.cmd") command<-paste(scriptAddr,"-h") shell(command,"cmd")
scriptAddr<-"path_to_install_stacomi.cmd" (e.g. "C:/Users/Public/stacomir/Installation/install_stacomi.cmd") command<-paste(scriptAddr,"-H hostname -P port -u user -p password")
command<-paste(scriptAddr,"-P 6543 -u foo -p foofoo") #I don't need to add -H localhost as it is the default settings shell(command,"cmd")
I tried to clarify in the documentation the admin rights for the SQL user. If you made the postgreSQL installation yourself it is the username/password you provided during the installation. If you didn't install the program by yourself you need to ask the person who did it to give you a username with the rights to create a database and a connection role. Indeed to make graphs with the stacomiR using the graphical interface provided by the package you need to have data in a database. This database will be created and fill with a sample set of data automatically with the cmd program but to do this you really have to have a username with the right to create a database. The user you create during the installation of PostgreSQL have all the rights needeed.
The difficulty with the installation is that we have to connect R with other programs like postgreSQL. In France and Spain, several structures use this program and as these users often have no knowledge neither in R nor in database, we provide a support of help to all the users by realizing ourselves the installation of everything that is needed on their computers. Thanks to our wish to publish our package in JOSS and the comments of our 2 reviewers, we have done a lot of work to simplify the installation. But actually, the user must always have some knowledge including how PostgreSQL works if he wants to be autonomous in the installation (otherwise, he must contact us so we can help). We can hardly see how to do otherwise. Indeed, if the user does not know if postgreSQL is installed on his computer on a localhost or on a distant server or does not have in mind the user and/or the password provided during the installation of postgres this can cause difficulties during installation. In our cmd program we have entered default values ββfor the server (= localhost), for the port (= 5432 i.e. the default port on which postgresql is installed) and for user and password (postgres for both). But the user will certainly have to change at least the user and the password (except if he chose this user and password during the installation). This is possible either with the command prompt or now in R with the shell function. We hope that all these informations will help. @boshek: don't hesitate to contact us if there are still things that are not clear. We will be happy to help you. We spent lot of time trying to improve our package and we really hope to publish it in JOSS
Hi @MarionLegrandLogrami - just a quick check in here to ask if you've considered putting all this into a docker container which can reproducibly run. That was you'd only have to provide instructions to the user on how to use docker. How feasible do you think that would be?
Hi @boshek - Thank you for your message. We spent some time reading a little about the docker container with which we are not familiar. This technology seems very interesting, but we are not sure that it will facilitate the installation (as it seem to be not really easy to use docker especially if the goal is to run an R package with a graphical interface - how can you access the graphical interface and interact with it inside the docker is still a mistery to me). What about the shell() command we suggested to use to launch the installation inside R ? Have you had any difficulties ? Have you managed to install postgresql ? If you encounter difficulties with any part of the installation don't hesitate to contact us. We will be very happy to help you (for example by using Teamviewer to help you to install everything)
Hi @MarionLegrandLogrami - I still think that the whole script can be abstracted away into an R function and the script itself could be included in the package in the inst
folder. Arguments to think function could be the username, password, host and port. That function would run the script for you. I think if you asked users if they were ok with the function running an executable you'd be ok.
In terms of the process, I am finally able to get this up and running. I think you can still improve the documentation by separating out the odbc section. However it is SIGNIFICANTLY improved. I am of course now experiencing R issues. When I tested the odbc connection everything was good. Here is my output and session info:
> library(stacomiR)
Loading required package: stacomirtools
Loading required package: RODBC
Registered S3 methods overwritten by 'ggplot2':
method from
[.quosures rlang
c.quosures rlang
print.quosures rlang
> Sys.setenv(LANG = "en")
> stacomi()
Error in funout(msg3, arret = TRUE) : connection impossible
In addition: Warning messages:
1: In RODBC::odbcDriverConnect("DSN=bd_contmig_nat;UID=iav;PWD=iav", :
[RODBC] ERROR: state IM014, code 0, message [Microsoft][ODBC Driver Manager] The specified DSN contains an architecture mismatch between the Driver and Application
2: In RODBC::odbcDriverConnect("DSN=bd_contmig_nat;UID=iav;PWD=iav", :
ODBC connection failed
> devtools::session_info()
- Session info ----------------------------------------------------------------------------------------------------------------------------
setting value
version R version 3.6.0 (2019-04-26)
os Windows 10 x64
system x86_64, mingw32
ui RStudio
language (EN)
collate English_United States.1252
ctype English_United States.1252
tz America/Los_Angeles
date 2019-05-10
- Packages --------------------------------------------------------------------------------------------------------------------------------
package * version date lib source
acepack 1.4.1 2016-10-29 [1] CRAN (R 3.6.0)
assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.6.0)
backports 1.1.4 2019-04-10 [1] CRAN (R 3.6.0)
base64enc 0.1-3 2015-07-28 [1] CRAN (R 3.6.0)
bit 1.1-14 2018-05-29 [1] CRAN (R 3.6.0)
bit64 0.9-7 2017-05-08 [1] CRAN (R 3.6.0)
blob 1.1.1 2018-03-25 [1] CRAN (R 3.6.0)
cairoDevice 2.26 2019-03-21 [1] CRAN (R 3.6.0)
callr 3.2.0 2019-03-15 [1] CRAN (R 3.6.0)
checkmate 1.9.3 2019-05-03 [1] CRAN (R 3.6.0)
chron 2.3-53 2018-09-09 [1] CRAN (R 3.6.0)
cli 1.1.0 2019-03-19 [1] CRAN (R 3.6.0)
cluster 2.0.8 2019-04-05 [2] CRAN (R 3.6.0)
colorspace 1.4-1 2019-03-18 [1] CRAN (R 3.6.0)
crayon 1.3.4 2017-09-16 [1] CRAN (R 3.6.0)
data.table 1.12.2 2019-04-07 [1] CRAN (R 3.6.0)
DBI 1.0.0 2018-05-02 [1] CRAN (R 3.6.0)
desc 1.2.0 2018-05-01 [1] CRAN (R 3.6.0)
devtools 2.0.2 2019-04-08 [1] CRAN (R 3.6.0)
digest 0.6.18 2018-10-10 [1] CRAN (R 3.6.0)
dplyr 0.8.0.1 2019-02-15 [1] CRAN (R 3.6.0)
foreign 0.8-71 2018-07-20 [2] CRAN (R 3.6.0)
Formula 1.2-3 2018-05-03 [1] CRAN (R 3.6.0)
fs 1.3.0 2019-05-02 [1] CRAN (R 3.6.0)
ggplot2 3.1.1 2019-04-07 [1] CRAN (R 3.6.0)
glue 1.3.1 2019-03-12 [1] CRAN (R 3.6.0)
gridExtra 2.3 2017-09-09 [1] CRAN (R 3.6.0)
gsubfn 0.7 2018-03-16 [1] CRAN (R 3.6.0)
gtable 0.3.0 2019-03-25 [1] CRAN (R 3.6.0)
gWidgets 0.0-52 2013-02-12 [1] R-Forge (R 3.6.0)
gWidgetsRGtk2 0.0-86 2018-03-01 [1] CRAN (R 3.6.0)
Hmisc 4.2-0 2019-01-26 [1] CRAN (R 3.6.0)
htmlTable 1.13.1 2019-01-07 [1] CRAN (R 3.6.0)
htmltools 0.3.6 2017-04-28 [1] CRAN (R 3.6.0)
htmlwidgets 1.3 2018-09-30 [1] CRAN (R 3.6.0)
intervals 0.14.0 2013-03-09 [1] R-Forge (R 3.6.0)
knitr 1.22 2019-03-08 [1] CRAN (R 3.6.0)
lattice 0.20-38 2018-11-04 [2] CRAN (R 3.6.0)
latticeExtra 0.6-28 2016-02-09 [1] CRAN (R 3.6.0)
lazyeval 0.2.2 2019-03-15 [1] CRAN (R 3.6.0)
lubridate 1.7.4 2018-04-11 [1] CRAN (R 3.6.0)
magrittr 1.5 2014-11-22 [1] CRAN (R 3.6.0)
Matrix 1.2-17 2019-03-22 [2] CRAN (R 3.6.0)
memoise 1.1.0 2017-04-21 [1] CRAN (R 3.6.0)
mgcv 1.8-28 2019-03-21 [2] CRAN (R 3.6.0)
munsell 0.5.0 2018-06-12 [1] CRAN (R 3.6.0)
nlme 3.1-139 2019-04-09 [2] CRAN (R 3.6.0)
nnet 7.3-12 2016-02-02 [2] CRAN (R 3.6.0)
pillar 1.3.1 2018-12-15 [1] CRAN (R 3.6.0)
pkgbuild 1.0.3 2019-03-20 [1] CRAN (R 3.6.0)
pkgconfig 2.0.2 2018-08-16 [1] CRAN (R 3.6.0)
pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.6.0)
plyr 1.8.4 2016-06-08 [1] CRAN (R 3.6.0)
prettyunits 1.0.2 2015-07-13 [1] CRAN (R 3.6.0)
processx 3.3.0 2019-03-10 [1] CRAN (R 3.6.0)
proto 1.0.0 2016-10-29 [1] CRAN (R 3.6.0)
ps 1.3.0 2018-12-21 [1] CRAN (R 3.6.0)
purrr 0.3.2 2019-03-15 [1] CRAN (R 3.6.0)
R6 2.4.0 2019-02-14 [1] CRAN (R 3.6.0)
RColorBrewer 1.1-2 2014-12-07 [1] CRAN (R 3.6.0)
Rcpp 1.0.1 2019-03-17 [1] CRAN (R 3.6.0)
remotes 2.0.4 2019-04-10 [1] CRAN (R 3.6.0)
reshape2 1.4.3 2017-12-11 [1] CRAN (R 3.6.0)
RGtk2 2.20.36 2019-03-08 [1] CRAN (R 3.6.0)
rlang 0.3.4 2019-04-07 [1] CRAN (R 3.6.0)
RODBC * 1.3-15 2017-04-13 [1] CRAN (R 3.6.0)
rpart 4.1-15 2019-04-12 [2] CRAN (R 3.6.0)
RPostgreSQL 0.6-2 2017-06-24 [1] CRAN (R 3.6.0)
rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.6.0)
RSQLite 2.1.1 2018-05-06 [1] CRAN (R 3.6.0)
rstudioapi 0.10 2019-03-19 [1] CRAN (R 3.6.0)
scales 1.0.0 2018-08-09 [1] CRAN (R 3.6.0)
sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.6.0)
sqldf 0.4-11 2017-06-28 [1] CRAN (R 3.6.0)
stacomiR * 0.5.4.2 2019-03-06 [1] CRAN (R 3.6.0)
stacomirtools * 0.5.3 2018-10-06 [1] CRAN (R 3.6.0)
stringi 1.4.3 2019-03-12 [1] CRAN (R 3.6.0)
stringr 1.4.0 2019-02-10 [1] CRAN (R 3.6.0)
survival 2.44-1.1 2019-04-01 [2] CRAN (R 3.6.0)
tibble 2.1.1 2019-03-16 [1] CRAN (R 3.6.0)
tidyselect 0.2.5 2018-10-11 [1] CRAN (R 3.6.0)
usethis 1.5.0 2019-04-07 [1] CRAN (R 3.6.0)
withr 2.1.2 2018-03-15 [1] CRAN (R 3.6.0)
xfun 0.6 2019-04-02 [1] CRAN (R 3.6.0)
xtable 1.8-3 2018-09-21 [1] R-Forge (R 3.6.0)
[1] C:/Users/boshek/Documents/R/win-library/3.6
[2] C:/Program Files/R/R-3.6.0/library
Hi @boshek - thank you for this detailed feedback on the problems encountered during installation. Depending on the error message, the problem is that you have installed an odbc driver that does not rely on the same architecture than the one used by R (you have a 64-bit R version and a 32-bit ODBC driver). There are 2 solutions:
Regarding your first suggestion (include everything in the package), we unfortunately can not follow it because for the package to be accepted by CRAN, it is necessary that the data does not exceed 5MB.
The compressed database is already 8 MB. If we try to include it in the package, we may not be able to submit it to CRAN. (read on the CRAN Repository Policy pages)
Your feedback on the quality of the documentation seems very important for us. Unfortunately, I am not sure I understand the suggestion you made to improve the documentation. At this point, I have separated the odbc driver installation detail in a specific part of the documentation (part 6. More help), leaving only in the "Requirement" part the indication that you need to have an odbc driver and indicating a link to the specific part of the document in which I explain how to make this installation. You propose to separate this part. You mean to extract it completely from the documentation to put it in another help file? If you could give me some details I will be very grateful
Hello @boshek - Did you manage to install the 64-bit postgresql driver or did you try the 32-bit R version compatible with the postgresql driver that you already installed? Please, let me know if you have any problem !
π @boshek - any news on this review?
@danielskatz sorry Daniel I haven't been able to prioritize this. I am very much still interested but my foreseable calendar is pretty full. Thank you for pinging me. I will endeavour to re-test this.
@boshek can you let us know when you might be able to complete your review? It's ok if you need more time, but a target date would be helpful.
@kyleniemeyer - Dear Editor, we are very grateful for the time spent by our reviewers on the submission of our package to JOSS and for their comments. Thanks to them we made lot of progress ! I was planning to use this paper as part of my Phd Thesis for which time is running out as we are waiting to see our package accepted by JOSS. This is a package that has passed all R CRAN checks and is already used by many organizations on a daily basis, and we are freely and actively advising and helping end users, mostly people doing fish migration monitoring in the field. To be fully functional, the package requires a database connection which makes the installation more complex. We offered our reviewer to help him install the various necessary programs and also developed a specific module to help him install postgres and set up the connection. To date we have trouble understanding where he got troubles in the installation. So we are coming back to you to see if it could be possible to speed up the validation process of our package in JOSS ? Again we are at your disposal and that of the reviewer to do anything to help you. Thanking you in advance for your help.
@MarionLegrandLogrami - I've just emailed @boshek to ask if they can come and take another look at this review soon.
Ok I will commit to having this finished by the end of August 9th. Sorry to have languished so long here. I still think that too much complexity is being offloaded to the user here rather than abstracting that away.
I am actively working on this but the end of August 9th might not be possible. I will update my status at the end of August 9th.
This is an update to this process.
I was running into issues because of leftover installations and other uses of a local PostgreSQL server. Additionally I was running into issues because I had installed the ODBC driver via the StackBuilder. That installed the driver in a different place than manually installing like the instructions said. I would recommend having your installation script check both locations for the driver.
That said I have been able to only go a little further in the process in getting the package/database up and running. I am able to successfully bring up the UI using stacomi()
. I do get this error however:
Loading required package: stacomirtools
Loading required package: RODBC
> stacomi()
C:/program files/stacomi/calcmig.csv does not exist, switching to defaut package fileLoading required package: gWidgetsRGtk2
Loading required package: RGtk2
Loading required package: gWidgets
Loading required package: cairoDevice
That said everything proceed in the sense that the dashboard appears. However I am not able to make it do anything (generate figures or tables) and I am assuming that the connection to the database isn't functioning properly. My first thought was to manually connect to the database in R and check if any actual data is there:
library(DBI)
library(dplyr)
#>
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#>
#> filter, lag
#> The following objects are masked from 'package:base':
#>
#> intersect, setdiff, setequal, union
library(dbplyr)
#>
#> Attaching package: 'dbplyr'
#> The following objects are masked from 'package:dplyr':
#>
#> ident, sql
con <- dbConnect(RPostgres::Postgres(),dbname = 'bd_contmig_nat',
host = 'localhost',
port = 5432,
user = 'postgres',
password = 'foo')
dbListTables(con)
#> [1] "chevauchement"
tbl(con, in_schema("iav", "t_bilanmigrationjournalier_bjo_bjo_identifiant_seq"))
#> # Source: table<iav.t_bilanmigrationjournalier_bjo_bjo_identifiant_seq>
#> # [?? x 3]
#> # Database: postgres [postgres@localhost:5432/bd_contmig_nat]
#> last_value log_cnt is_called
#> <int64> <int64> <lgl>
#> 1 351677 0 TRUE
Created on 2019-08-09 by the reprex package (v0.3.0)
I am unclear that this output is telling me and whether it is helpful at all. However at this point this is as far as I could get. I thought to try installing the dev version but got some GTK errors which I am unable to debug at this point:
Warning: Failed to load RGtk2 dynamic library, attempting to install it. Please install GTK+ from http://ftp.gnome.org/pub/gnome/binaries/win32/gtk+/2.22/gtk+-bundle_2.22.1-20101227_win32.zip If the package still does not load, please ensure that GTK+ is installed and that it is on your PATH environment variable IN ANY CASE, RESTART R BEFORE TRYING TO LOAD THE PACKAGE AGAIN Error in inDL(x, as.logical(local), as.logical(now), ...) : unable to load shared object 'C:/Users/samal/Documents/R/win-library/3.6/RGtk2/libs/i386/RGtk2.dll': LoadLibrary failure: The specified module could not be found.
I still think that the installation script could be written completely in R avoiding the clunky step of downloading the installation file and running it. I think that you have misunderstood my meaning here. I don't want to run the installation script with R. I think it should be written in R itself so that a user can debug in the language they are most likely to be familiar with - R. This would provide a unified interface for users. However, I think given the length of this process and admitting that reasonable people can disagree on this point, this is not a point that is worth holding up acceptance of this review particularly since my more material concerns have been ameliorated by the process that the installation script enacts and very thorough documentation.
After finally being able to generate the UI for this process I can't help but ask why you aren't doing this in Shiny. This comment is only relevant to this review because it would great ease a user experience (mine being documented here) taking the target audience for a dashboard (non R users) and taking them completely out of R. Without even considering the much improved UI experience, Shiny would be far easier for you to deploy. For a future I would strongly recommend using Shiny for the functionality currently being filled for GTK+.
This review has been going on for a very long time now. I'm very grateful to both @steffilazerte and @boshek for their work here - I believe the stacomiR
package is in substantially better shape because of their time and efforts.
While I believe there are still some challenges with this software, I'm going to make the editorial decision at this point to accept this submission into JOSS.
@MarionLegrandLogrami - At this point could you make a new release of this software that includes the changes that have resulted from this review. Then, please make an archive of the software in Zenodo/figshare/other service and update this thread with the DOI of the archive? For the Zenodo/figshare archive, please make sure that:
I can then move forward with accepting the submission.
@whedon add @boshek as reviewer
OK, @boshek is now a reviewer
@whedon add @boshek as reviewer
@boshek - this is just to make sure you're listed at the top of this review thread. There's nothing new to do here.
Thank you very much for accepting this submission in JOSS and @steffilazerte and @boshek - thanks again for your time and advice!
@arfon - Here's the DOI (for the archive of the package): http://doi.org/10.5281/zenodo.3371259
@whedon set 10.5281/zenodo.3371259 as archive
OK. 10.5281/zenodo.3371259 is the archive.
@whedon accept
Attempting dry run of processing paper acceptance...
Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/907
If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/907, then you can now move forward with accepting the submission by compiling again with the flag deposit=true
e.g.
@whedon accept deposit=true
@whedon accept deposit=true
Doing it live! Attempting automated processing of paper acceptance...
π¦π¦π¦ π Tweet for this paper π π¦π¦π¦
π¨π¨π¨ THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! π¨π¨π¨
Here's what you must now do:
Party like you just published a paper! πππ¦ππ»π€
Any issues? notify your editorial technical team...
@steffilazerte @boshek, many thanks for your reviews here.
@MarionLegrandLogrami - your paper is now accepted into JOSS :zap::rocket::boom:
:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:
If you would like to include a link to your paper from your README use the following code snippets:
Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.00791/status.svg)](https://doi.org/10.21105/joss.00791)
HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.00791">
<img src="https://joss.theoj.org/papers/10.21105/joss.00791/status.svg" alt="DOI badge" >
</a>
reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.00791/status.svg
:target: https://doi.org/10.21105/joss.00791
This is how it will look in your documentation:
We need your help!
Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:
Submitting author: @MarionLegrandLogrami (Marion Legrand) Repository: https://github.com/MarionLegrandLogrami/stacomiR Version: 0.5.3.1 Editor: @arfon Reviewers: @steffilazerte, @boshek Archive: 10.5281/zenodo.3371259
Status
Status badge code:
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@steffilazerte, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:
The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @arfon know.
Review checklist for @steffilazerte
Conflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper
paper.md
file include a list of authors with their affiliations?