![]() But in both cases performance tends to plateau very quickly and throwing more of the same at it (more threads, bigger buffers, etc.) doesn't make much difference.Īlso keep in mind that the numbers you get from ccsiobench are based on a relatively short IO bursts. Conversely, IO buffer sizes and counts matter when copying very large files. ![]() I suspect that you did, but it won't hurt to ask.īased on my own testing I can say that the copying thread count (exec threads) has the biggest impact when copying lots of smaller files and when creating or deleting folders in bulk. Have you confirmed that your overrides were actually picked up and used by the program? As recorded in the backup log. Bvckup2 doesn't have any special processing of the last file block when the file is opened in this mode (though this is _now_ on the todo list). This is not an issue when reading, but when writing it may cause an error if the file size is NOT a sector-size multiple. In particular, keep in mind the sector-multiple requirement of the "no-buffering" mode. The defaults are 28000000 for "src" and 08000000 for "dst", or "no buffering + sequential" and "sequential" respectively. random-access and write-through will be A0000000 (because 2 + 8 in hex is A). To specify a combo you need to add respective flag values, e.g. With the latter permitted only in the "dst" value. These are bitfields (bit combinations) of the following flags. Mar 23, 2021This is controlled via the following two settings. If you have time, I'd be interested to see what you find out. But if you feel like experimenting, relevant entries are It's basically smart enough to not interfere with the process in a bad way. In other words, I'd keep threading setup at its defaults. So its benefits when copying larger files are really quite negligible. ![]() At least not for long.Īlso, with parallel exec the main speed benefit comes from parallelizing non-IO operations - file creation, meta info copying, etc. This means that even though both thread counts can be set to ~ # of CPU cores, in practice both thread sets - the hashing and the execution - will never be active together. The idea here is that if the parallel exec runs into a file that needs delta copying, it won't start any other copies until this - "heavy" - copy operation completes. One controls the number of hashing threads in the delta copying mode and another the number of copying threads when parallel exec is enabled. Threading - there are two separate thread counts. It kicks in when the file is smaller than 32 full IO buffers, so that'd be 256MB in case of remote copying. There is a separate IO config for files that are copied in a so-called "light" mode. By default, extra_bufs are set so that the total number of buffers is at least 4. This is done because in the delta copying mode every block also goes through the hashing phase, so the whole thing can juggle more blocks in-flight than the straight copying. If a file is to be copied in delta copying mode, then the buffer count is increased by It is after all derived from an internal profiling tool used to settle on Bvckup2 IO defaults. 32 x 256KB or similar.ĬcsioBench may in fact provide a good reference here. You may want to try bumping 8 to 16 or splitting total buffer space of 8MB differently, e.g. The default for copying to/from a remote share is 8 x 1MB. It reads in full buffers and it writes in full buffers (except for the last file block and cases when a part of the block is detected as unchanged by the delta copying logic).īuffer size/count for non-delta copying is defined by There is no separate control over read vs write sizes. You can indeed vary buffer sizes and buffer counts. So the first thing to do is to look at your numbers.Ģ. This happens because delta copying eliminates most of the writes, so the copying _is_ in fact read-bound. In my example, the bottleneck is in reading even though the backup goes from a local SSD to a NAS over an ordinary 1 Gbps link. Instead, they are meant for relative comparison between themselves. ![]() If you are copying files in full, you will see only two numbers there - for reading and writing. The last three numbers are performance estimates of three parts of the delta copying process - blocks are read, then hashed and checked for changes, and then they are written out. It will be something like thisĬompleted in 10 min 33 sec, copied 5.18 GB out of 256 GBĤ13.93 MBps | 414.03 reading, 1770.54 hashing, 1194.02 writing Pick a large file from the backup log and look at its "Completed in" line. ![]()
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |