Significantly increase BinTubes speed by using memory instread of HDD
Increasing speed and reducing HDD traffic is required for people with one slow HDD and a fast (e.g. 120Mbit) internet connection. I have seen the processing take 3 times as long as the downloading itself. I have two ideas for doing this for a standard rarred release (many rar files in which the relase is stored and not compressed):
- Load all parts of each downloaded file (up to a certain file size limit) into memory instead of HDD and decode it to memory. (I guess memory required is 3 times the file size because we need to store the parts of file 1, the decoded file, and parts of file 2 when decoding file 1)
- the second item is to unrar directly from memory and not store the rar file at all. When repairing is required, repair the extracted file directly by using a similar sceme as the ReScene Data Recovery Archive tool uses. What I mean is when loading the par2 blocks, add the rar file headers on the fly, and write the repaired block directly to the unrarred file (without the rar headers off course)
These ideas would reduce data being written and read to the harddisk by a factor of 3 and thus increase the speed by a factor of 3 if the HDD is limiting. I suggest to make it an option with a "maximum memory to use" parameter.
We’ve added a Buffering option that should dramatically increase performance on HDD bound computers.
Please update to the latest version.