Hardware RAID-6 constitutes a significant (and unnecessary) portion of investment to storage system.
Many hardware RAID controllers offer RAID-5 but not RAID-6. However RAID-5 is not suitable for storage systems due to its poor reliability.
Whenever required storage capacity is up to or exceed the combined capacity of 2 HDDs RAID6 is a solution to eliminate downtime in case of HDD failure. With redundancy enough to survive loss of one or two disks RAID6 is a critical technology for storage systems.
Drawbacks of hardware RAID
However even best hardware RAID controllers have drawbacks that make software RAID more attractive.
Enterprise class hardware storage systems and generally available NAS devices share the same weaknesses and limitations that come from fundamental design concepts and do not depend on price or vendor.
More disaster recovery planning
Hardware RAID controllers have serious implications for disaster recovery planning.
In case of controller failure the downtime is clearly inevitable but the actual issue is a lack of control. When controller is dead the only option is to wait for replacement which may not be immediately available. Actual down time may depend on quality of vendor's support, support contract, couriers delivery times etc.
Loss of array configuration
Replacement of RAID controllers may not be a trivial matter. Consider the situation when old controller had different firmware than the new one. There is a risk that new controller may not pick up array configuration properly. Although such situation is improbable it is also may not be well tested. Ideally to avoid troubles broken hardware shall be replaced with exactly the the same parts and firmware revisions due to risks or incompatible RAID configuration.
Lack of upgrade options
Hardware failure is an opportunity for upgrade. Old hardware may be expensive to replace when the same amount of money could buy something better. While investment to obsolete hardware is a waste of resources there are no guarantees that some years later new model of RAID controller will recognise old array configuration.
Poor value for money
Controllers with on-board memory are not very effective: onboard memory is often redundant as Operating System is already caching the same frequently accessible regions. Arguably more system memory will be better investment.
Operating system uses RAM more effectively and intelligently.
Hardware write-back is bad for reliability
If RAID controller is buffering writes it is lying to operating system that data is flushed to disk(s) when in fact data may be still in controller's RAM. Unlike operating system controller doesn't make a difference between data and journal so it can't decide what should be flushed first. This is a significant risk for file system's integrity in case of power loss or crash of operating system: data considered as saved by the operating system may be inconsistent if flushing sequence was disturbed by controller.
Untested components vs. reliability
Reliability is a matter of trust to all RAID components after they are putted to test.
Onboard controller's RAM is rarely tested, if ever. Onboard memory may be difficult (if possible) to test. This is another weakness and risk for data corruption. Does your RAID controller support ECC RAM? Can it be trusted?
Battery is never tested in controllers with battery-backed RAM. How much time will it buy you in case of power loss before fatal discharge and potential loss of data? Battery voltage monitoring is poor indication of capacity as it gives no information regarding speed of discharge.
Software components
Hardware RAID is never fully hardware: how do you know when one HDD
failed in the array? Software mdadm RAID will send you an email. If
hardware RAID is capable of sending emails it is likely to be
implemented with installable software counterpart that may not be
supported on your operating system not to mention that it could be
proprietary.
Hidden HDD interface
Often hardware controller hides low-level interface to HDDs and appears as virtual device to operating system. This may be convenient for booting from RAID-5 or RAID-6 but it is quite bad for HDD health monitoring due to lack of access to S.M.A.R.T. interface.
Firmware
Controller's firmware means "lack of testing" if developed in closed proprietary manner. Is there public bug tracker for firmware? Is there an easy way to share a feedback with vendor and more importantly is it worth the time spent?
Infrequent firmware updates and lack of testing should be expected for hardware RAID.
Perhaps proprietary nature of RAID controllers is a biggest problem of all.
Lock-in
Number of issues are associated with vendor lock-in.
HDDs and server from single vendor.
Often through the terms of warranty vendors insist that HDDs are to be ordered from the same vendor as server.
In most cases such restrictions unnecessarily limit the choice of available models and therefore may be considered as intimidation.
Overpriced HDDs.
If you ever considered buying server from vendors like Intel or HP you may know that their HDDs are ridiculously expensive and priced several times higher than ordinary HDD of similar capacity that generally available from Seagate/Hitachi/Samsung/Western Digital or other vendors.
Expensive disks do not necessarily live longer. Even if support plan oblige vendor to promptly ship the replacement for failed HDD using courier delivery it doesn't make sense to invest in such support when the price of one such HDD can buy few spare HDDs for immediate substitution. Like expensive insurance does not reduce probability of accident, expensive support plan does not contribute to reliability or minimising down time.
Conclusion
Software RAID implemented with mdadm have number of advantages over
hardware RAID solutions.