LEVEL 1: No Deduplication (Compression and Hashing Only)
Use Level 1 deduplication when your backup device used to store the Unitrends backups and database partition cannot sustain the IO needed for greater levels, or when only absolute minimum retention (2 full backup rotations) would be used rendering the need for deduplication negligible. This setting trades storage needs for performance, allowing customers to use slower performance storage at lower prices where long term retention in the backup device of data is not a concern. This is also the option you should use if you intend on having more than one storage device added to CONFIGURE > Storage tab. It ensures that your selection for the storage location is not compromised by deduplicated blocks that could be on any of the storage devices.
Storage size needs are greatest when using Level 1 deduplication as every backup will require it's full free space allocation minus compression. The functional minimum requirement for storage is 2X the size of the protected data set, and 3X or more is typically required to store 30+ days of backups. Increasing retention further requires linear increases in storage size to accommodate. The appliance will store every block received independently.
Understand, the largest consumer of IO in a Unitrends System is the database. The database can be relocated in virtual machines to independent higher performance storage without the investment in high performance storage for the entire unit. Please see the KB Move the Unitrends database off of stateless storage and onto a new partition
. If your DB can be moved to storage of not less than 500 write IOPS, (2K or more IOPS may be required for larger appliances) then it is often possible to run at LEVEL 3 on backup storage that might only otherwise support LEVEL 1. Note:
On units that are not also replicating Hot Copy backups, when using LEVEL 1 Deduplication, customer are advised to also disable the following setting: Configure > Appliances > Select Your Appliance > Click Edit > Advanced Tab > Click General Configuration then configure the following entry: Section InlineHashing > Name hashFileContents > Value = No. This will further decrease database overhead and improve performance. This setting should not be disabled if Hot Copy Replication is used. LEVEL 2: Post Process Deduplication (Compression First, Deduplication Later)
In Level 2, the entire process of deduplication occurs as a passive background operation on backup sets that are not the current backup set, and blocks that exist in more than 1 backup are moved to Single Instance Storage (SIS). To reach this deep deduplication, your storage devices needs to be at least 2.5x your protected data set size. It will typically require 3X your protected capacity size to reach 30 days of retention similar to LEVEL 1, but beyond this value, additional retention is gained by only adding new change data in addition and is no longer linear. This system is best targeted at customers who retain not less than 3 full backup rotations, and the greatest benefit will be seen for those 6 or more full backup sets. The appliance will store typically 2 copies of each unique block, plus requires additional write storage overhead for post process deduplication that can be as much as the largest individual backup. If database performance is constrained, additional backups may be maintained in non-deduplciated states increasing storage requirements.
Use Level 2 to improve performance under the following conditions:
1) The hardware used for the UB is not robust enough to handle the workload of LEVEL 3 inline deduplication but retention of more than 3 complete backup sets is needed and storage capacity will not accommodate LEVEL 1 to meet those needs.
2) Your wish full system restore performance to be a primary focus, but are unable to meet SLAs using replicas or other methods for instant recovery and must rely on full traditional restores. This mode ensures data is recovered the fastest in those cases.
3) Your Cold Backup Copies (previously known as Archives) take too long to complete to meet offsite SLAs, and other optimizations for Cold Copy have already been reviewed and are still not sufficient, and Hot Copy replication is not an option to meet those offsite SLAs.
LEVEL 3: Advanced Adaptive Deduplication (Deduplication on Arrival and Synthesis)
Though this model greatly reduces database IO requirements for an appliance, it actually can increase general backup storage IO requirements somewhat. It also has the highest functionally minimum acceptable storage size due to the unique overhead of this mode.
Anytime a Unitrends Database can be relocated to alternate storage to improve performance to allow LEVEL 3, that is recommended before choosing this mode, and it may still be recommended when using LEVEL 2 in scenarios where restore performance is still a customer priority.
Deduplication occurs as the data is received, and all block storage is sent directly to the SIS directory for backup types that support deduplication. This method has the lowest storage size requirements, but has the highest system performance requirements as a trade-off. Level 3 will typically sustain 30 days of retention with as little as 1.5X protected capacity, and additional retention only ever requires space to store new unique changed data. The appliance will only ever store 1 copy of each unique block. Using this method may also slightly reduce backup performance and restore performance. The impact on performance is primarily affected by the performance of hardware provided to the Unitrends Database in terms of IO speed, CPU, and Memory. Initial first full backups may typically take 2-4X normal expectations, however due to overhead reductions of deduplciation of future backups, UB systems deployed to best practices will typically perform future fulls faster than when not using deduplciation.
Customers wishing to use this mode are encouraged to deploy UB systems with equivalent hardware configurations to Unitrends Physical appliances that would be sold for similar data sets, and should follow Deployment Best Practices for Unitrends Backup
. Customers deploying systems to protect 5TB or more of data will have the best cost vs performance by ensuring high performance storage is available to the unitrends Database engine. High performance storage is not required for bulk block storage, in fact, block storage for LEVEL 3 independent of the database has the lowest IO needs of any mode.
This mode is required to use Unitrends Image backup types introduced in 10.3.0 and is required on Hot Copy target systems. It is the intended mode when using Long Term Retention beyond 90 days. Using a NAS as your Storage Device
Though many NAS devices may internally support reasonable IO performance, network layers between your virtual appliance and your NAS device typically inhibit IO response times dramatically, even on 10G networks, and may not be suitable for running the Unitrends Database Engine for LEVEL 2 or LEVEL 3 use except in the smallest environments. Your Unitrends Onboarding Engineer or a Support Customer Engineer can measure the performance of any intended storage system and advise where the unitrends Database is constrained, and assist in moving it to alternate storage, or, assist in providing guidance for proper mode of deduplciation and other options above that would improve performance. For Backup Storage use, Unitrends Supports NAS devices configured connected to VM hosts that those hypervisor vendors themselves have certified when configured as datasrtores and when using unitrends best Practices of attaching virtual disks in storage to the UB. Support for CIFS/NFS passthrough is clarified in this KB: Supported external storage vendors for use with Unitrends Backup appliances