caffe fork runtest failed

Hi, I want to post this in your caffe fork but there is no issue tab there.

So, I have tried to compile your latest caffe fork and encounter some failure when running `make runtest' there.

Do you have any suggestion? Perhaps something like revert to the stable commit or your DenseBlock_release1 will do?

The failure happen when testing DenseBlock layer and

[ RUN      ] DenseBlockLayerTest/0.TestDenseBlockBwd
/home/amax/ardiya/dncaffe/src/caffe/test/test_DenseBlock_layer.cpp:317: Failure
The difference between filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) and filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) is   0.55306005477905273, which exceeds 0.4, where
filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to 0.15142345428466797,
filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to 0.7044835090637207, and
0.4 evaluates to 0.40000000000000002.
/home/amax/ardiya/dncaffe/src/caffe/test/test_DenseBlock_layer.cpp:317: Failure
The difference between filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) and filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) is   0.54173290729522705, which exceeds 0.4, where
filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to -1.1120017766952515,
filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to -1.6537346839904785, and
0.4 evaluates to 0.40000000000000002.
/home/amax/ardiya/dncaffe/src/caffe/test/test_DenseBlock_layer.cpp:317: Failure
The difference between filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) and filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) is   1.1154916286468506, which exceeds 0.4, where
filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to -1.2618510723114014,
filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to -0.14635944366455078, and
0.4 evaluates to 0.40000000000000002.
/home/amax/ardiya/dncaffe/src/caffe/test/test_DenseBlock_layer.cpp:317: Failure
The difference between filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) and filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) is   0.5621497631072998, which exceeds 0.4, where
filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to -1.9996230602264404,
filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to -1.4374732971191406, and
0.4 evaluates to 0.40000000000000002.
/home/amax/ardiya/dncaffe/src/caffe/test/test_DenseBlock_layer.cpp:336: Failure
The difference between layer3localScaler->diff_at(0,channelIdx,0,0) and layer4localScaler->diff_at(0,channelIdx,0,0) is 1.100369930267334, which exceeds 0  .4, where
layer3localScaler->diff_at(0,channelIdx,0,0) evaluates to 4.863670825958252,
layer4localScaler->diff_at(0,channelIdx,0,0) evaluates to 3.763300895690918, and
0.4 evaluates to 0.40000000000000002.
/home/amax/ardiya/dncaffe/src/caffe/test/test_DenseBlock_layer.cpp:345: Failure
The difference between layer3localBias->diff_at(0,channelIdx,0,0) and layer4localBias->diff_at(0,channelIdx,0,0) is 0.55823898315429688, which exceeds 0.4  , where
layer3localBias->diff_at(0,channelIdx,0,0) evaluates to 38.147781372070312,
layer4localBias->diff_at(0,channelIdx,0,0) evaluates to 37.589542388916016, and
0.4 evaluates to 0.40000000000000002.
[  FAILED  ] DenseBlockLayerTest/0.TestDenseBlockBwd, where TypeParam = caffe::CPUDevice<float> (751 ms)

and the data layer.

[ RUN      ] DataLayerTest/0.TestReadCropTestLevelDB
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:229: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 1
Expected: i
Which is: 0
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:229: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 0
Expected: i
Which is: 1
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:229: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 4
Expected: i
Which is: 2
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:229: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 2
Expected: i
Which is: 4
[  FAILED  ] DataLayerTest/0.TestReadCropTestLevelDB, where TypeParam = caPUDevice<float> (282 ms)
[ RUN      ] DataLayerTest/0.TestSkipLevelDB
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 1
Expected: label % batch_size
Which is: 3
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 0
Expected: label % batch_size
Which is: 1
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 4
Expected: label % batch_size
Which is: 3
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 3
Expected: label % batch_size
Which is: 1
...........(omitted)
[  FAILED  ] DataLayerTest/0.TestReadCropTestLevelDB, where TypeParam = caPUDevice<float> (282 ms)

I tried to train it on imagenet, and the data layer failed to load lmdb. Any idea?

lmdb test fixed by changing random_order in caffe.proto to optional bool random_order = 11 [default = false]; still not sure about the backpropagation error in denseblock layer

Hello @ardiya , for the first question which is failure of unit test about DenseBlock, one strange part is there is the numbers on both sides seem to be large integers, so what is the order of instructions runned? And what is your environment? Thanks.

Hi @Tongcheng, thanks for replying.

one strange part is there is the numbers on both sides seem to be large integers

It is a float number, but there might be some problem when I copied it. Here is the pastebin link for a better viewing https://pastebin.com/Jh8iQeNT

so what is the order of instructions runned?

I clone it, removeCmakeDep.sh, mkdir build, cd build, cmake .., make -j8 and make runtest -j8

what is your environment?

I am using Ubuntu 14.04.1 and this is the result of cmake .. https://pastebin.com/yujC0Ry5 Please tell me if you need more info, because I really have no idea what should I write here.

I encounter same problem when i run make runtest. Can anyone help me? thanks ` [----------] Global test environment tear-down [==========] 1957 tests from 268 test cases ran. (313258 ms total) [ PASSED ] 1945 tests. [ FAILED ] 12 tests, listed below: [ FAILED ] DenseBlockLayerTest/0.TestDenseBlockBwd, where TypeParam = caffe::CPUDevice [ FAILED ] DenseBlockLayerTest/1.TestDenseBlockBwd, where TypeParam = caffe::CPUDevice [ FAILED ] DenseBlockLayerTest/2.TestDenseBlockBwd, where TypeParam = caffe::GPUDevice [ FAILED ] DenseBlockLayerTest/3.TestDenseBlockBwd, where TypeParam = caffe::GPUDevice [ FAILED ] NesterovSolverTest/0.TestLeastSquaresUpdateWithMomentumMultiIter, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/0.TestNesterovLeastSquaresUpdateWithEverythingShare, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/0.TestNesterovLeastSquaresUpdateWithMomentum, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/0.TestNesterovLeastSquaresUpdateWithEverything, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/1.TestNesterovLeastSquaresUpdateWithMomentum, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/1.TestNesterovLeastSquaresUpdateWithEverythingShare, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/1.TestNesterovLeastSquaresUpdateWithEverything, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/1.TestLeastSquaresUpdateWithMomentumMultiIter, where TypeParam = caffe::CPUDevice

12 FAILED TESTS make[3]: [src/caffe/test/CMakeFiles/runtest] 错误 1 make[2]: [src/caffe/test/CMakeFiles/runtest.dir/all] 错误 2 make[1]: [src/caffe/test/CMakeFiles/runtest.dir/rule] 错误 2 make: [runtest] 错误 2 `

I got the same problem. I think I did the same procedure as ardiya did. I cloned https://github.com/Tongcheng/caffe.git. Then, run removeCmakeDep.sh, mkdir build, cd build, cmake .., make -j4 and make runtest -j4. I am running Ubuntu 16.04, CUDA 8.0, cuDnn v5.1.

Hi, @ardiya I suffer from similar problem, have you solved this? thanks.

Hi, @Tongcheng @ardiya @zhimengfan1990 I meet the same problem while running 'make runtest', have you solved this? thanks.

%% ...... /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 4 Expected: label % batch_size Which is: 3 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 1 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 0 Expected: label % batch_size Which is: 4 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 2 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 2 Expected: label % batch_size Which is: 0 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 0 Expected: label % batch_size Which is: 3 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 0 Expected: label % batch_size Which is: 1 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 2 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 0 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 4 Expected: label % batch_size Which is: 0 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 1 Expected: label % batch_size Which is: 3 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 1 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 1 Expected: label % batch_size Which is: 4 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 2 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 1 Expected: label % batch_size Which is: 0 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 0 Expected: label % batch_size Which is: 3 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 2 Expected: label % batch_size Which is: 1 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 4 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 2 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 1 Expected: label % batch_size Which is: 0 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 0 Expected: label % batch_size Which is: 3 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 4 Expected: label % batch_size Which is: 1 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 2 Expected: label % batch_size Which is: 4 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 4 Expected: label % batch_size Which is: 2 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 1 Expected: label % batch_size Which is: 3 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 4 Expected: label % batch_size Which is: 1 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 2 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 1 Expected: label % batch_size Which is: 0 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 1 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 4 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 2 Expected: label % batch_size Which is: 3 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 4 Expected: label % batch_size Which is: 1 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 1 Expected: label % batch_size Which is: 4 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 2 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 4 Expected: label % batch_size Which is: 0 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 1 Expected: label % batch_size Which is: 3 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 1 Expected: label % batch_size Which is: 4 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 0 Expected: label % batch_size Which is: 2 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 1 Expected: label % batch_size Which is: 0 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 4 Expected: label % batch_size Which is: 3 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 1 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 0 Expected: label % batch_size Which is: 4 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 0 Expected: label % batch_size Which is: 2 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 0 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 4 Expected: label % batch_size Which is: 3 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 0 Expected: label % batch_size Which is: 1 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 4 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 3 Expected: label % batch_size Which is: 2 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 1 Expected: label % batch_size Which is: 3 /home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure Value of: blob_toplabel->cpu_data()[i] Actual: 4 Expected: label % batch_size Which is: 1 .....

Hi, @zhangleiedu, I meet the same problem? have you solve this problem? thanks! %% [ FAILED ] 12 tests, listed below: [ FAILED ] NesterovSolverTest/0.TestNesterovLeastSquaresUpdateWithMomentum, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/0.TestNesterovLeastSquaresUpdateWithEverythingShare, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/0.TestLeastSquaresUpdateWithMomentumMultiIter, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/0.TestNesterovLeastSquaresUpdateWithEverything, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/1.TestNesterovLeastSquaresUpdateWithEverythingShare, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/1.TestLeastSquaresUpdateWithMomentumMultiIter, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/1.TestNesterovLeastSquaresUpdateWithMomentum, where TypeParam = caffe::CPUDevice [ FAILED ] NesterovSolverTest/1.TestNesterovLeastSquaresUpdateWithEverything, where TypeParam = caffe::CPUDevice [ FAILED ] DenseBlockLayerTest/0.TestDenseBlockBwd, where TypeParam = caffe::CPUDevice [ FAILED ] DenseBlockLayerTest/1.TestDenseBlockBwd, where TypeParam = caffe::CPUDevice [ FAILED ] DenseBlockLayerTest/2.TestDenseBlockBwd, where TypeParam = caffe::GPUDevice [ FAILED ] DenseBlockLayerTest/3.TestDenseBlockBwd, where TypeParam = caffe::GPUDevice

hello @ardiya @Tongcheng I meet exactly the same problem, have you solved it? Despite the unsettled problem, I still successfully trained my own model based on Dense_Block, but when I tested the model using python interface, it occured that:

WARNING: Logging before InitGoogleLogging() is written to STDERR W0504 11:01:52.860939 8205 _caffe.cpp:139] DEPRECATION WARNING - deprecated use of Python interface W0504 11:01:52.860967 8205 _caffe.cpp:140] Use this instead (with the named "weights" parameter): W0504 11:01:52.860972 8205 _caffe.cpp:142] Net('/home/sunyu/caffe-master/data/mydata/deploy.prototxt', 1, weights='/home/sunyu/caffe-master/data/mydata/Dense_EXP_1/my_regression_iter_35000.caffemodel') [libprotobuf ERROR google/protobuf/text_format.cc:274] Error parsing text-format caffe.NetParameter: 56:20: Message type "caffe.LayerParameter" has no field named "denseblock_param". F0504 11:01:52.862316 8205 upgrade_proto.cpp:90] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /home/sunyu/caffe-master/data/mydata/deploy.prototxt

I just cannot figure out that why "caffe.LayerParameter" has no field named "denseblock_param" only during the test phase while trainning is ok?

Tongcheng / DN_CaffeScript

caffe fork runtest failed #2