NetBackup Hyper-V backups with Equallogic result in error 56

I've spent the better part of the last two days working with a co-worker who is a Hyper-V administrator trying to figure out why backing up VMs on his new Server 2012 R2 Hyper-V cluster were failing with error 156. We initially were using the default (auto) snapshot provider type, and that failed, showing a number of errors in the Hyper-V host's log relating to SNMP being unable to communicate with Physical Disks in the SAN. 1stError-WhenVDS-VSS-accessNotEnabled

This clued us in that it must be attempting to use snapshots at the array level rather than in the operating system. Changing the option to "Software" worked some of the time but still resulted in occasional 156 errors, changing it to "System" failed completely. But we opted not to engage Microsoft on this, but rather to try to solve the hardware-based snapshots first.

Today, he contacted Equallogic support, and together properly configured the Equallogic's support for VSS. Notably, enabling VSS access, and setting access permissions from "Volume Only" to "Volumes and Snapshots". That cleared up the error below:

errorOnEQLwhenAccessPolicyOnlyAppliedToVolumesNotVolsAndSnapshots

In addition, the Host Integration Toolkit was upgraded to 4.7. After that was done, setting the snapshot type back to "auto" was successful, and performance is quite good, with the 1Gbps Hyper-V host NIC acting as the bottleneck for the backups.

Watching in the Equallogic management console, you can clearly see the snapshots being created at the array level during backups now.

I thought I'd post this, as administering vSphere on Equallogic, I would not have expected the array to get itself involved in backup operations unless explicitly configured, but apparently it does for Hyper-V deployed against Equallogic, at least when using NetBackup 7.6.0.2 with the snapshot provider type set to auto.

How to install NetBackup 7.0 on Ubuntu 10.04

NetBackup 7 officially “supports” Ubuntu Linux 8.04, 9.04, and 10.04 for Linux clients, as seen in the Software Compatibility List but the installer doesn’t “work”.

Specifically, the problem seems to be that PBX, or Private Branch Exchange, is distributed in an RPM package along with the client.  The normal shell script that installs the client kicks an error when PBX fails to install:

Installing PBX... ./installpbx: 1510: rpm: not found
 ERROR: Could not install VRTSpbx package
 Please see installation log for more details
 Installation log located here: 
/var/tmp/installpbx-1928-100810135211.log

 Installing PBX was unsuccessful.
 Aborting ...

Opening the next log reveals more:

[10-08-10-13:52:11] Extracting 
/var/tmp/VRTSpbx_1928/PBX.tar.gz into 
/var/tmp/VRTSpbx_1928
[10-08-10-13:52:11] Installing/Upgrading private branch 
exchange on Linux
[10-08-10-13:52:11] Checking for the PBX process...
[10-08-10-13:52:11] PBX process is not running.
[10-08-10-13:52:11] rpm -U --nodeps --nopostun --nopreun
/var/tmp/VRTSpbx_1928/VRTSpbx-1.4.0.10-10.RH_x86_64.rpm
./installpbx: 1568: rpm: not found
[10-08-10-13:52:11] ERROR: Could not install VRTSpbx 
package
[10-08-10-13:52:11] Removing temporary directory
[10-08-10-13:52:11] rm -rf /var/tmp/VRTSpbx_1928
[10-08-10-13:52:11] installpbx exiting with return 
code: 1
[10-08-10-13:52:11] Please see installation log for more 
details
[10-08-10-13:52:11] Installation log located here:
/var/tmp/installpbx-1928-100810135211.log

As this isn’t Red Hat, rpm is not present on the system, nor would it work if it was…

With the help of the folks at Symantec, here’s what will get around this problem and help you install the client.  Hopefully they’ll put this online in a tech note soon, but I’m guessing I’m not the only person who ran into this problem, so here are my notes.  I am not a Linux wizard, by any means, but here are the required steps to make this work.  This also requires you to install alien, a package that’ll let you install rpm packages on Ubuntu:

 

Get the giant 7.0 client tarball and copy it to your server.  I downloaded mine from FileConnect.

gunzip NetBackup_7.0_CLIENTS_GA.tar.gz.gz

tar xvf NetBackup_7.0_CLIENTS_GA.tar.gz

cd NB_7.0_CLIENTS_GA/NBClients/anb/Clients/usr/openv/netbackup/client/Linux/RedHat2.6

sudo mkdir /tmp/pbx

sudo cp PBX.tar.gz /tmp/pbx

cd /tmp/pbx

sudo gzip -dv PBX.tar.gz

sudo tar xvf PBX.tar

sudo apt-get install alien

sudo alien -i --scripts VRTSpbx-1.4.0.10-10.RH_x86_64.rpm
(you will see lots of errors, you can ignore them)

Then start PBX on the CLIENT:
sudo /opt/VRTSpbx/bin/vxpbx_exchanged start

sudo nano NB_7.0_CLIENTS_GA/NBClients/anb/Clients/usr/openv/netbackup/client/Linux/RedHat2.6/cp_to_client

Comment out lines 1723 - 1732 by inserting a # at the start of the line (use Control-W in nano to search for PBX to help locate this):

  1723 #       ${ECHO} " 
  1724 #Installing PBX..." 
  1725 
  1726 #       ${SOURCE_DIR}/installpbx -f PBX.tar.gz 
  1727 #       if [ $? -ne 0 ] ; then 
  1728                 ${ECHO} " 
  1729 #Installing PBX was unsuccessful. 
  1730 #Aborting ..." 
  1731 #               exit 1 
  1732 #       fi

Hit Control-X to save and exit.

Then execute the "install" script in /NB_7.0_CLIENTS_GA

Success!

You can then normally patch the client to version 7.01, as that won’t require you to use any RPM-based packages.  I've also heard that installing version 6.5.4, then upgrading from there to 7.0 will work too, as that version of PBX will install ok, but I haven’t tried it.