For speed + compression level, plzip – parallel lzip implementation seems best but memory consumption is out of this world at 188x to 270x times more memory used than gzip!.But lzip takes much longer to run and consumes up to 40x times more memory. That’s nearly 1/3rd of the compressed size for gzip with 22% at level 3 and 19% at levels 6 to 9. For compression and smaller compressed file sizes, lzip seems to be best being able to compress MySQL sql backup file to 7% of original file size at level 3 compression.Updated: Check test #4 below for details or jump straight to results. Tweaking the block size for compression can reduce the max memory usage while still retaining similar compression ratios and fast compression times. More physical cpu cores you’d have, the faster the speed up for pigz vs gzip. Probably limited by the fact that the test server has only 2 physical cpu cores. For speed seems pigz is best up to 2x times faster than gzip.Original uncompressed file: MySQL backup sql file sized at 605,530 KB at /root/sqlbackup/vb387.sqlįrom the below tests, you can see how much faster pigz is than gzip.Xeon W3540 – 4 core allocation (2 physical + 2 virtual).VirtualBox CentOS server instance might not produce the most accurate timer stats for the tests, but it should give you an idea of speed of compression, cpu and memory usage for various compression methods outlined above. plzip v0.80 rc1 – multi-threaded version of lzip.lzip v1.1.3 rc1 – based on LZMA compression algorithm.pbzip2 v1.1.6 – multi-threaded version of bzip2.pigz v2.1.6 – multi-threaded version of gzip.I cleaned up my gziptest.sh script and decided to run some linux compression benchmark tests comparing the following compression methods: Pigz definitely does speed up compression times, I implemented parallel multi-threaded versions of gzip and bzip2 – pigz and pbzip2 in my mysqlmybackup.sh script I am writing so have seen first hand the speed up benefits. Update: Part 2 compression benchmarks have been posted here. A Cpanel feature request thread came up for inclusion for pigz a parallel multi-threaded implementation of gzip which dramatically speeds up compression.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |