Feature requests for elasticity solver

ahojukka5 commented 9 years ago

Let's put here all feature requests regarding to elasticity solver (this is a good start of writing features.md)

[x] Linear and quadratic elements
[x] Finite displacement
[ ] Non-linear material models: hyperelastic materials
[ ] Non-linear material models: incremental material models
[ ] Contact mechanics: surface to surface (Mortar method)
[ ] Possibility to give own subroutines: UMAT
[ ] Dual Lagrange boundary condition (http://www.code-aster.org/V2/doc/default/en/man_r/r3/r3.03.01.pdf)

TeroFrondelius commented 9 years ago

@ahojukka5 we have an existing features.md

ahojukka5 commented 9 years ago

I think it's a little mislabeled. We don't actually have those features implemented yet, it's more like a road map. My goal in this issue is to have a discussion board where we can follow the progress of development and share ideas and opinions about features and their implementations.

TeroFrondelius commented 9 years ago

Here is a good learning site: http://solidmechanics.org/FEA.htm There is also a full example code: http://solidmechanics.org/FEM_MATLAB/FEM_2Dor3D_viscoplastic.m

ahojukka5 commented 9 years ago

In principle here's the code

  E = 1/2*(delta_u' + delta_u + delta_u'*delta_u)  # Green-Lagrange strain tensor
  S = lambda*trace(E)*I + 2*mu*E  # PK2 stress tensor
  P = F*S  # PK1 stress tensor

Where must

calculate principal components of deformation gradient to get different strain measures (Log or Hencky strain in ABAQUS), GL strain tensor is easy to compute because it doesn't need polar decomposition,
create function stress!(args..) returning stress for given strain measure including some internal variables, now we use linear material model P=F_(lambda_trace(E)_I + 2_mu*E) without any internal parameters

Log-strain is known to perform well when we have large strains (ABAQUS theory manual?).

Or course this needs also cumbersome modifications to local stiffness matrix linearisation but luckily @jvaara has volunteered for that :).

ahojukka5 commented 9 years ago

3d_solid_model

3d elasticity with linear material and nonlinear geometry.

ovainola commented 9 years ago

:+1:

ahojukka5 commented 9 years ago

I was thinking a bit how do we actually want to create model and make solutions, see elasticity solver notebook. It's unfinished but should give some ideas. Main concern is what is a nice syntax to define dirichlet and neumann boundary conditions. I basically want them to be defined to model so that they can be saved with model before solution. Boundary conditions can be defined to elements, boundary elements or nodes. Boundary elements can be implicitly defined like is done in ABAQUS by defining e.g. element with element side number of explicitly like is done in e.g. Elmer and Code Aster. In implementation level I think we finally end up to define some field variable to element nodes and integrate over domain. Dirichlet boundary conditions are defined in nodes while neumann bc is basically always defined in elements (nodal load is a special case neumann bc defined to 0-dimension element assigned to vertex). It's probably a good practice to always set boundary conditions to sets, because in real problems we don't want to set things node or element-wise but to sets of elements or nodes.

TeroFrondelius commented 9 years ago

Depending how we will define adaptive mesh, but the nodes and elements should be able to change. Making the finer mesh is typically no problem, but if you want to make coarser mesh, then your area changes from original. Many commercial code use geometry for loads and boundary conditions.

ahojukka5 commented 9 years ago

Consider for example following situations:

want to add normal direction pressure load to one surface of element, varying from 1 to 2 along boundary
want to set displacement for node 1, dx=0, dy=1 and no restrictions in dz
want to have line load in z direction for 3d element boundary between nodes 1 and 2
want to have dx=dy=dz for nodes in set "SUPPORT"
want to set flux in temperature problem to 1 in elements in set "LOAD"
want to set pressure load for element set "LOAD" varying in each element, may depends even from z direction (hydrostatic pressure, for example)

All of these cases finally ends up in matrix level to either manipulate stiffness matrix for essential boundary conditions or modify right hand side for natural boundary conditions. Main question is how to efficiently handle all kind of special cases so that everything is possible.

Maybe I should do some test cases using Code Aster to get some ideas

ahojukka5 commented 9 years ago

Btw, are we even interested about h-refined adaptive meshes? We can always rise the order of shape functions if we use hierarchical basis (=> p-refined mesh). I recently read about so called "patch methods", where more refined mesh is added to the interesting area and "glued" to the actual mesh using mortar method. It seems to be very promising approach to solve very small details in very large models, see introduction chapter of http://www.unige.ch/~gander/Preprints/gander_mini_4.pdf and references there. Refining meshes by splitting elements sounds.. old-fashioned.

If we are satisfied to the approximation properties of linear tetras to describe the model geometry I don't even see much of point to use 2nd order Lagrange elements. Using hierarchical (P=2) basis the stiffness system is better conditioned with approximately same approximation properties and therefore easier to solve when using iterative solvers. Side nodes adds unnecessary complexity in model if they are not really used to model curved surfaces. At least I don't figure out any good reasons to have side nodes if they are not used to anything useful like describing the geometry of the model. But I might be wrong, I'm not an expert of hierarchical basis...

ahojukka5 commented 9 years ago

Piston, undeformed and deformed configuration

piston

ahojukka5 commented 8 years ago

Some performance results. I was able to run 226 kdof model with my 8 GB RAM PC. CPU usage was 300-400 % typically.

Model 1 -- piston_8789_P1.inp (26367 dofs, 37331 elements)

INFO: # of field problems: 1
INFO: # of boundary problems: 1
INFO: Starting iteration 1
INFO: # of dofs: 26367, # of interface dofs: 1350
INFO: solved. solution norm: 1.6019130688664045
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 1.4992079734802246
INFO: field assembly         : 17.662078142166138
INFO: create sparse matrices : 0.35225915908813477
INFO: solution of system     : 3.4736850261688232
INFO: update element data    : 3.5246031284332275
INFO: non-linear iteration   : 26.51184892654419
INFO: Starting iteration 2
INFO: # of dofs: 26367, # of interface dofs: 1350
INFO: solved. solution norm: 0.0067431681280311255
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.10223507881164551
INFO: field assembly         : 15.527868032455444
INFO: create sparse matrices : 0.09523797035217285
INFO: solution of system     : 2.3252041339874268
INFO: update element data    : 3.429192066192627
INFO: non-linear iteration   : 21.479762077331543
INFO: Starting iteration 3
INFO: # of dofs: 26367, # of interface dofs: 1350
INFO: solved. solution norm: 3.7777213379512435e-6
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.09183979034423828
INFO: field assembly         : 14.835246801376343
INFO: create sparse matrices : 0.09740090370178223
INFO: solution of system     : 2.1218039989471436
INFO: update element data    : 3.3479089736938477
INFO: non-linear iteration   : 20.494215965270996
INFO: Starting iteration 4
INFO: # of dofs: 26367, # of interface dofs: 1350
INFO: solved. solution norm: 1.1213470006791607e-13
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.0926828384399414
INFO: field assembly         : 15.927090883255005
INFO: create sparse matrices : 0.09426212310791016
INFO: solution of system     : 2.151948928833008
INFO: update element data    : 3.351346969604492
INFO: non-linear iteration   : 21.617354154586792
INFO: solver finished in 91.42705702781677 seconds.

Model 2 -- piston_16436_P1.inp (49308 dofs, 72801 elements)

INFO: # of field problems: 1
INFO: # of boundary problems: 1
INFO: Starting iteration 1
INFO: # of dofs: 49308, # of interface dofs: 2241
INFO: solved. solution norm: 3.3143208191999682
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 1.5976500511169434
INFO: field assembly         : 33.965296030044556
INFO: create sparse matrices : 0.5512678623199463
INFO: solution of system     : 10.744216918945312
INFO: update element data    : 11.934814929962158
INFO: non-linear iteration   : 58.79326105117798
INFO: Starting iteration 2
INFO: # of dofs: 49308, # of interface dofs: 2241
INFO: solved. solution norm: 0.022013526109564686
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.16490411758422852
INFO: field assembly         : 31.45194101333618
INFO: create sparse matrices : 0.332578182220459
INFO: solution of system     : 11.741114139556885
INFO: update element data    : 12.016391038894653
INFO: non-linear iteration   : 55.70695090293884
INFO: Starting iteration 3
INFO: # of dofs: 49308, # of interface dofs: 2241
INFO: solved. solution norm: 3.1695786266265306e-5
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.18024396896362305
INFO: field assembly         : 34.35774278640747
INFO: create sparse matrices : 0.3308279514312744
INFO: solution of system     : 10.30850887298584
INFO: update element data    : 11.898698091506958
INFO: non-linear iteration   : 57.07604002952576
INFO: Starting iteration 4
INFO: # of dofs: 49308, # of interface dofs: 2241
INFO: solved. solution norm: 5.778099535259944e-12
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.17047691345214844
INFO: field assembly         : 35.704561948776245
INFO: create sparse matrices : 0.32314014434814453
INFO: solution of system     : 10.487448930740356
INFO: update element data    : 11.96686315536499
INFO: non-linear iteration   : 58.65250587463379
INFO: solver finished in 233.86731004714966 seconds.

Model 3 -- piston_27343_P1 (82029 dofs)

INFO: # of field problems: 1
INFO: # of boundary problems: 1
INFO: Starting iteration 1
INFO: # of dofs: 82029, # of interface dofs: 3474
INFO: solved. solution norm: 5.527830089669539
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 1.6489348411560059
INFO: field assembly         : 55.22538709640503
INFO: create sparse matrices : 1.100071907043457
INFO: solution of system     : 26.08756709098816
INFO: update element data    : 32.53634786605835
INFO: non-linear iteration   : 116.59832715988159
INFO: Starting iteration 2
INFO: # of dofs: 82029, # of interface dofs: 3474
INFO: solved. solution norm: 0.04828924879598316
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.2585930824279785
INFO: field assembly         : 53.973127126693726
INFO: create sparse matrices : 0.8619270324707031
INFO: solution of system     : 27.616240978240967
INFO: update element data    : 32.49492692947388
INFO: non-linear iteration   : 115.20483303070068
INFO: Starting iteration 3
INFO: # of dofs: 82029, # of interface dofs: 3474
INFO: solved. solution norm: 0.0001236166189635111
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.27105283737182617
INFO: field assembly         : 55.63293480873108
INFO: create sparse matrices : 0.8651039600372314
INFO: solution of system     : 30.244996070861816
INFO: update element data    : 33.100571155548096
INFO: non-linear iteration   : 120.11467695236206
INFO: Starting iteration 4
INFO: # of dofs: 82029, # of interface dofs: 3474
INFO: solved. solution norm: 6.798894059224977e-11
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.27588415145874023
INFO: field assembly         : 62.89068794250488
INFO: create sparse matrices : 0.843101978302002
INFO: solution of system     : 30.530904054641724
INFO: update element data    : 32.684978008270264
INFO: non-linear iteration   : 127.22558403015137
INFO: solver finished in 488.18594098091125 seconds.

Model 4 -- piston_45510_P1.inp (136530 dofs, 216713 elements)

max. memory usage 64 %

INFO: # of field problems: 1
INFO: # of boundary problems: 1
INFO: Starting iteration 1
INFO: # of dofs: 136530, # of interface dofs: 4836
INFO: solved. solution norm: 9.321000744005222
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 1.7502689361572266
INFO: field assembly         : 101.91970896720886
INFO: create sparse matrices : 1.6882209777832031
INFO: solution of system     : 63.30455303192139
INFO: update element data    : 94.63053798675537
INFO: non-linear iteration   : 263.2938470840454
INFO: Starting iteration 2
INFO: # of dofs: 136530, # of interface dofs: 4836
INFO: solved. solution norm: 0.1084202414780616
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.7207770347595215
INFO: field assembly         : 118.50161600112915
INFO: create sparse matrices : 1.614767074584961
INFO: solution of system     : 81.57157301902771
INFO: update element data    : 97.2911069393158
INFO: non-linear iteration   : 299.6998679637909
INFO: Starting iteration 3
INFO: # of dofs: 136530, # of interface dofs: 4836
INFO: solved. solution norm: 0.0005151019107847139
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.7267799377441406
INFO: field assembly         : 120.79992008209229
INFO: create sparse matrices : 1.390895128250122
INFO: solution of system     : 88.15509486198425
INFO: update element data    : 96.08679509162903
INFO: non-linear iteration   : 307.1595048904419
INFO: Starting iteration 4
INFO: # of dofs: 136530, # of interface dofs: 4836
INFO: solved. solution norm: 8.996433828538284e-10
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.7029261589050293
INFO: field assembly         : 109.79263687133789
INFO: create sparse matrices : 1.3379099369049072
INFO: solution of system     : 84.32997298240662
INFO: update element data    : 96.54092907905579
INFO: non-linear iteration   : 292.70439410209656
INFO: solver finished in 1186.5935790538788 seconds.

Model 5 -- piston_75470_P1.inp (226410 dofs, 365531 elements)

max. memory usage 85 % CPU 100 %, mem 25 % -- update ? CPU 300 %, mem 30 % -- assembly ? PCU 400 %, mem 85 % -- solution ?

INFO: # of field problems: 1
INFO: # of boundary problems: 1
INFO: Starting iteration 1
INFO: # of dofs: 226410, # of interface dofs: 7317
INFO: solved. solution norm: 15.789332396913549
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 2.452112913131714
INFO: field assembly         : 182.9222388267517
INFO: create sparse matrices : 2.892953872680664
INFO: solution of system     : 149.12483096122742
INFO: update element data    : 266.96219301223755
INFO: non-linear iteration   : 604.354560136795
INFO: Starting iteration 2
INFO: # of dofs: 226410, # of interface dofs: 7317
INFO: solved. solution norm: 0.24832979966255178
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.62117600440979
INFO: field assembly         : 197.51101183891296
INFO: create sparse matrices : 2.9609899520874023
INFO: solution of system     : 208.66468787193298
INFO: update element data    : 267.09473991394043
INFO: non-linear iteration   : 676.8530480861664
INFO: Starting iteration 3
INFO: # of dofs: 226410, # of interface dofs: 7317
INFO: solved. solution norm: 0.002353362095676507
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.6154818534851074
INFO: field assembly         : 200.55590510368347
INFO: create sparse matrices : 3.091750144958496
INFO: solution of system     : 201.07792901992798
INFO: update element data    : 268.8166949748993
INFO: non-linear iteration   : 674.1583919525146
INFO: Starting iteration 4
INFO: # of dofs: 226410, # of interface dofs: 7317
INFO: solved. solution norm: 1.4580592297477459e-8
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.6126217842102051
INFO: field assembly         : 203.64583802223206
INFO: create sparse matrices : 2.9024770259857178
INFO: solution of system     : 201.25202703475952
INFO: update element data    : 274.0600199699402
INFO: non-linear iteration   : 682.4730179309845
INFO: solver finished in 2700.233761072159 seconds.

ahojukka5 commented 8 years ago

It seems that CPU usage is 300 % during assembling and 400 % during solution of system. Max memory usage ~85 % in this model. Element data update no longer takes time after a little modification to solver code. CPU time is used approx. 50-50 for assembly and for solution when calculating models of size 100-200 kdof. When model size increases the most of the time is used in solving the system of equations.

Model 6 -- piston_55950_P2.inp (167850 dofs, 37331 elements)

INFO: # of field problems: 1
INFO: # of boundary problems: 1
INFO: Starting iteration 1
INFO: Assembling problems.
INFO: # of dofs: 167850, # of interface dofs: 5100
INFO: Solving system
INFO: solved. solution norm: 5.860119119800823
INFO: Updating element data
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 2.0514039993286133
INFO: field assembly         : 95.13076877593994
INFO: create sparse matrices : 1.2354719638824463
INFO: solution of system     : 91.88808012008667
INFO: update element data    : 0.5261378288269043
INFO: non-linear iteration   : 190.83503603935242
INFO: Starting iteration 2
INFO: Assembling problems.
INFO: # of dofs: 167850, # of interface dofs: 5100
INFO: Solving system
INFO: solved. solution norm: 0.04356616854547006
INFO: Updating element data
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.5794951915740967
INFO: field assembly         : 112.7850980758667
INFO: create sparse matrices : 1.3376049995422363
INFO: solution of system     : 102.07598304748535
INFO: update element data    : 0.494365930557251
INFO: non-linear iteration   : 217.27306079864502
INFO: Starting iteration 3
INFO: Assembling problems.
INFO: # of dofs: 167850, # of interface dofs: 5100
INFO: Solving system
INFO: solved. solution norm: 9.917942286768701e-5
INFO: Updating element data
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.5832958221435547
INFO: field assembly         : 114.09875392913818
INFO: create sparse matrices : 1.3200359344482422
INFO: solution of system     : 97.45724892616272
INFO: update element data    : 0.49151086807250977
INFO: non-linear iteration   : 213.95136284828186
INFO: Starting iteration 4
INFO: Assembling problems.
INFO: # of dofs: 167850, # of interface dofs: 5100
INFO: Solving system
INFO: solved. solution norm: 3.265679159398905e-11
INFO: Updating element data
INFO: timing info for non-linear iteration:
INFO: boundary assembly      : 0.5989480018615723
INFO: field assembly         : 114.15645790100098
INFO: create sparse matrices : 1.3242409229278564
INFO: solution of system     : 95.77240300178528
INFO: update element data    : 0.4889061450958252
INFO: non-linear iteration   : 212.34146809577942
INFO: solver finished in 835.0044188499451 seconds.

ahojukka5 commented 8 years ago

Model 7 -- piston_107168_P2 (321504 dofs, 72801 elements)

Reducing stiffness matrix using static condensation to reduce memory usage. Memory usage ~80%. (8 GB memory in my nuhapumppu.)

INFO: # of field problems: 1
INFO: # of boundary problems: 1
INFO: Starting iteration 1
INFO: Assembling boundary problems...
INFO: # of interface dofs: 8568
INFO: Assembling field problems...
INFO: Assembling body 1...
INFO: # of dofs in problem 1: 321504
INFO: Eliminating interior dofs for body 1...
INFO: Solving interface system
INFO: Solved, calculating interior dofs...
INFO: Problem solved. solution norm: 11.391177449684704
INFO: timing info for non-linear iteration:
INFO: boundary assembly       : 2.3574090003967285
INFO: field assembly          : 244.46906685829163
INFO: reduce stiffness matrix : 2559.243628025055
INFO: create sparse matrices  : 2.9678919315338135
INFO: solution of system      : 117.18335199356079
INFO: update element data     : 2.174860954284668
INFO: non-linear iteration    : 2991.6151881217957
INFO: Starting iteration 2
INFO: Assembling boundary problems...
INFO: # of interface dofs: 8568
INFO: Assembling field problems...
INFO: Assembling body 1...
INFO: # of dofs in problem 1: 321504
INFO: Eliminating interior dofs for body 1...
INFO: Solving interface system
INFO: Solved, calculating interior dofs...
INFO: Problem solved. solution norm: 0.11391526987582962
INFO: timing info for non-linear iteration:
INFO: boundary assembly       : 1.2126209735870361
INFO: field assembly          : 253.97624897956848
INFO: reduce stiffness matrix : 2653.1463718414307
INFO: create sparse matrices  : 13.380634784698486
INFO: solution of system      : 122.79443979263306
INFO: update element data     : 2.3563590049743652
INFO: non-linear iteration    : 3114.8880569934845
INFO: Starting iteration 3
INFO: Assembling boundary problems...
INFO: # of interface dofs: 8568
INFO: Assembling field problems...
INFO: Assembling body 1...
INFO: # of dofs in problem 1: 321504
INFO: Eliminating interior dofs for body 1...
INFO: Solving interface system
INFO: Solved, calculating interior dofs...
INFO: Problem solved. solution norm: 0.0004840672223823747
INFO: timing info for non-linear iteration:
INFO: boundary assembly       : 1.2372231483459473
INFO: field assembly          : 263.24039912223816
INFO: reduce stiffness matrix : 2646.7302730083466
INFO: create sparse matrices  : 8.446307897567749
INFO: solution of system      : 119.12197685241699
INFO: update element data     : 2.6256539821624756
INFO: non-linear iteration    : 3107.0321249961853
INFO: Starting iteration 4
INFO: Assembling boundary problems...
INFO: # of interface dofs: 8568
INFO: Assembling field problems...
INFO: Assembling body 1...
INFO: # of dofs in problem 1: 321504
INFO: Eliminating interior dofs for body 1...
INFO: Solving interface system
INFO: Solved, calculating interior dofs...
INFO: Problem solved. solution norm: 5.318359498098401e-10
INFO: timing info for non-linear iteration:
INFO: boundary assembly       : 1.3649930953979492
INFO: field assembly          : 262.0752999782562
INFO: reduce stiffness matrix : 2688.6258659362793
INFO: create sparse matrices  : 6.7869179248809814
INFO: solution of system      : 114.25468301773071
INFO: update element data     : 2.861567974090576
INFO: non-linear iteration    : 3144.579080104828
INFO: solver finished in 12359.256505012512 seconds.

Here reduce stiffness matrix means creating interface problem by eliminating interior degrees of freedom (using Cholesky factorization). Solution of system is solving the interface problem using LU factorization. Interface problem dimension is 8568. Youngs modulus 210.0e3, Poissons ratio 0.3. Surface pressure (0.0, 0.0, -10.0) for BC1. DX=DY=DZ=0 for BC2. Norm of displacement vector: 11.49967445173282. u_min = (-0.0161234, -0.0219069, -0.140821). u_max = (0.0161174, 0.02244, 0.00516401) piston

ahojukka5 commented 8 years ago

Improved performance a bit.

INFO: # of field problems: 1
INFO: # of boundary problems: 1
INFO: Starting iteration 1
INFO: Assembling boundary problems...
INFO: # of interface dofs: 8568
INFO: Assembling field problems...
INFO: Assembling body 1...
INFO: # of dofs in problem 1: 321504
INFO: Solving system
INFO: Solved, calculating interior dofs...
INFO: Problem solved. solution norm: 11.391177449683427
INFO: timing info for non-linear iteration:
INFO: boundary assembly       : 3.686000108718872
INFO: field assembly          : 162.32500004768372
INFO: reduce stiffness matrix : 5.069999933242798
INFO: create sparse matrices  : 0.1399998664855957
INFO: dump matrices to disk   : 0.0
INFO: solve problem           : 29.73900008201599
INFO: back substitute         : 0.0
INFO: update element data     : 1.621999979019165
INFO: non-linear iteration    : 204.1119999885559
INFO: Starting iteration 2
INFO: Assembling boundary problems...
INFO: # of interface dofs: 8568
INFO: Assembling field problems...
INFO: Assembling body 1...
INFO: # of dofs in problem 1: 321504
INFO: Solving system
INFO: Solved, calculating interior dofs...
INFO: Problem solved. solution norm: 0.11391526987712108
INFO: timing info for non-linear iteration:
INFO: boundary assembly       : 1.9660000801086426
INFO: field assembly          : 160.2829999923706
INFO: reduce stiffness matrix : 5.057999849319458
INFO: create sparse matrices  : 0.1420001983642578
INFO: dump matrices to disk   : 0.0
INFO: solve problem           : 60.674999952316284
INFO: back substitute         : 0.0
INFO: update element data     : 1.6159999370574951
INFO: non-linear iteration    : 231.24799990653992
INFO: Starting iteration 3
INFO: Assembling boundary problems...
INFO: # of interface dofs: 8568
INFO: Assembling field problems...
INFO: Assembling body 1...
INFO: # of dofs in problem 1: 321504
INFO: Solving system
INFO: Solved, calculating interior dofs...
INFO: Problem solved. solution norm: 0.0004840672223822576
INFO: timing info for non-linear iteration:
INFO: boundary assembly       : 2.003999948501587
INFO: field assembly          : 168.06199979782104
INFO: reduce stiffness matrix : 5.039999961853027
INFO: create sparse matrices  : 0.14100003242492676
INFO: dump matrices to disk   : 0.0
INFO: solve problem           : 60.27800011634827
INFO: back substitute         : 0.0
INFO: update element data     : 1.61899995803833
INFO: non-linear iteration    : 238.61800003051758
INFO: Starting iteration 4
INFO: Assembling boundary problems...
INFO: # of interface dofs: 8568
INFO: Assembling field problems...
INFO: Assembling body 1...
INFO: # of dofs in problem 1: 321504
INFO: Solving system
INFO: Solved, calculating interior dofs...
INFO: Problem solved. solution norm: 5.318293313386342e-10
INFO: timing info for non-linear iteration:
INFO: boundary assembly       : 2.0319998264312744
INFO: field assembly          : 166.31699991226196
INFO: reduce stiffness matrix : 5.141999959945679
INFO: create sparse matrices  : 0.1419999599456787
INFO: dump matrices to disk   : 0.0
INFO: solve problem           : 60.019999980926514
INFO: back substitute         : 0.0
INFO: update element data     : 1.621999979019165
INFO: non-linear iteration    : 236.77899980545044
INFO: solver finished in 912.1029999256134 seconds.

ahojukka5 commented 8 years ago

INFO: timing info for non-linear iteration:
INFO: boundary assembly       : 0.5800380706787109
INFO: field assembly          : 70.33733201026917
INFO: dump matrices to disk   : 9.5367431640625e-7
INFO: solve problem           : 17.688688039779663
INFO: update element data     : 1.7101640701293945
INFO: non-linear iteration    : 90.31624603271484
...
INFO: solver finished in 472.49817419052124 seconds.

ahojukka5 commented 8 years ago

Here's one using PETSc interface. Speed isn't comparable but there is one totally unnecessary step of copying matrices to PETSc sparse format and preconditioning didn't work at the moment. These shortcomings can be expected to change soon, PETSc.jl is on active development now. And one huge benefit is that PETSc already supports MPI, so we get basically can support distributed computing via MPI using PETSc interface. I think PETSc has good direct solvers too. This was done using GMRES without preconditioning.

INFO: # of field problems: 1
INFO: # of boundary problems: 1
INFO: Starting iteration 1
INFO: Assembling field problems...
INFO: Assembling body 1...
INFO: Assembling boundary problems...
INFO: Assembling boundary 1...
INFO: dumping matrices to disk, file = matrices_simulation1_host_1_iteration_1.jld
INFO: Solving system
INFO: PETSc: all dofs = 321504
INFO: PETSc: interior dofs = 312936
INFO: PETSc: boundary dofs = 8568
INFO: PETSc: displacement on boundary solved.
INFO: PETSc: homogeneous dirichlet boundary
INFO: PETSc: 312936 interior dofs, assembling to PETSc Mat
INFO: PETSC: initialization of matrices in 249.37704920768738 seconds
INFO: PETSc: performing ksp GMRES solve
INFO: PETSc: finished ksp solve
INFO: PETSc: ksp info:
0

KSP Object: 1 MPI processes
  type: gmres
    GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
    GMRES: happy breakdown tolerance 1e-30
  maximum iterations=10000, initial guess is zero
  tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
  left preconditioning
  using PRECONDITIONED norm type for convergence test
PC Object: 1 MPI processes
  type: bjacobi
    block Jacobi: number of blocks = 1
    Local solve is same for all blocks, in the following KSP and PC objects:
    KSP Object:    (sub_)     1 MPI processes
      type: preonly
      maximum iterations=10000, initial guess is zero
      tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
      left preconditioning
      using NONE norm type for convergence test
    PC Object:    (sub_)     1 MPI processes
      type: ilu
        ILU: out-of-place factorization
        0 levels of fill
        tolerance for zero pivot 2.22045e-14
        matrix ordering: natural
        factor fill ratio given 1, needed 1
          Factored matrix follows:
            Mat Object:             1 MPI processes
              type: seqaij
              rows=312936, cols=312936
              package used to perform factorization: petsc
              total: nonzeros=2.31458e+07, allocated nonzeros=2.31458e+07
              total number of mallocs used during MatSetValues calls =0
                using I-node routines: found 108274 nodes, limit used is 5
      linear system matrix = precond matrix:
      Mat Object:       1 MPI processes
        type: seqaij
        rows=312936, cols=312936
        total: nonzeros=2.31458e+07, allocated nonzeros=1.07963e+08
        total number of mallocs used during MatSetValues calls =0
          using I-node routines: found 108274 nodes, limit used is 5
  linear system matrix = precond matrix:
  Mat Object:   1 MPI processes
    type: mpiaij
    rows=312936, cols=312936
    total: nonzeros=2.31458e+07, allocated nonzeros=1.07963e+08
    total number of mallocs used during MatSetValues calls =0
      using I-node (on process 0) routines: found 108274 nodes, limit used is 5

INFO: PETSc: solved in 564.9358458518982 seconds. norm = 11.390988778587532
INFO: timing info for non-linear iteration:
INFO: boundary assembly       : 0.7768311500549316
INFO: field assembly          : 82.6037950515747
INFO: dump matrices to disk   : 3.5442869663238525
INFO: solve problem           : 566.2912859916687
INFO: update element data     : 1.8357789516448975
INFO: non-linear iteration    : 655.0545618534088

TeroFrondelius commented 7 years ago

@ahojukka5 is this issue up to date?

ahojukka5 commented 7 years ago

I think it's a misusing of issue list to open issues that cannot be closed anyway. I write a short description what is currently implemented to the documentation and after that close this.

JuliaFEM / JuliaFEM.jl