Description of ideas about improving unit test speed.
Table of Contents
Let's take the erp5_base bt with testERP5Base, it takes :
- by default, 7.9s for importing, 14.2s fully installed, 65s for
- by activating zexp files (replace the "if 0" with "if 1" in BT.py)
3.8s for importing, 9.9s fully installed, 54s for testERP5Base (all zexp
- by not importing skins files (only import required ones at run time)
2.3s for importing, 7.1s fully installed, 41s for testERP5Base
- by not importing skins file and using zexp files
1.3s for importing, 6.1s fully installed, 38s for testERP5Base
(all zexp available)
- by not using the filesystem anylonger and importing
directly bt5 files as ZEXP from an existing instance, 0.3s for importing
5.1s fully installed.
- by default import, then only installing needed skin files (from bt5)
JPS: lazy install of skins seems to provide a speed gain of 1.3s. However, I do not understand why importing time for "not importing skins file and using zexp files" is lower
than importing time for "by activating zexp files"
Seb: This is because the code (wich is not well designed) that create zexp files cut the bt5 into many zexp files (for every python script, every page template...). So the test
was done by importing only one part of the bt.
It could be possible to make activities runing on several cores.
- import bt5 in one big zexp for all bt5 that didn't change
- optimise XML parsing by any way (cache/MD5)
- do lazy import, so import on bt5 only needed stuff
- do lazy installation, so do not install in each bt5 what we don't need
- speed up indexing, in general, and may be with some trick (like only for some unit test, do not catalog portal types,...). One possible trick could consist in dumping the
catalog of an existing installed instance then delete any UIDs which do not relate to any existing path. However, this is not really a safe approach.
- it must also be considered to make lazy installation persistent, in order to not reimport while using load for many unit test -> use ZEXP import approach from another instance,
it is both faster and fully consistent with ERP5 ZODB based configuration approach
- HOW TO MAKE IT COMPATIBLE WITH --update-only ?
The fastest test execution can be obtained by
1. Importing already parsed and built ZEXP from a running ERP5 instance instead of parsing text files on the filesystem (this also has the advantage of eliminating the issue of
synchronization between files and ZODB).
2. Adding a certain amount of 'lazy install' so that certain parts of an already imported business template which contains numerous objects are only installed if needed. Skins
are a good candidate.
3. Reduce the number of objects which are indexed, whenever this is not needed for a test (ex. do not index portal type actions, do not index python scripts).
4- Accelerate catalog, by any possible means, including by running tests on a ZEO cluster with concurrent activities.
Reduce the number of objects which are indexed : this is already done. By default portal types are not indexed. It can be indexed again with option
--enable_full_indexing=portal_types for runUnitTest
Optimise XML parsing by any way. This is already done by storing xml file in zexp format in a gdbm storage. You can use ERP5_BT5_CACHE=1 environment variable when runing test.