Intro
So this is going to be a quick blogpost. I often find myself in the situation that I need to update something within VMware. VMware Update Manager (VUM) of course seems to be the best solution for this. However, often I found myself also in the situation that VUM just doesn’t work and for some reason doesn’t want to stage, remediate and update the ESXi host. And often, this is also most of the time of the following underlying factors.
Let’s sum them up.
Issue 1: vmfs volume doensn’t have enough space
Often I find that one of the vmfs volumes has used a lot of all of his space. Now to be honest, the 80% is also after I cleaned it up earlier. I often see that one of the volumes is completely 100% used up. To get a good overview of the space usage on the ESXi host use the command (on the ESXi host):
df -h
If you want to see within the folder which files are using up the space, use the command:
Du -s -h *
Often, this is most of the time a log file that has been growing excessively, or certain downloads for VMware tools or other packages that have not been cleaned-up. So check it out, but make sure that you know that you are deleting the right stuff.
Extra Tip: When you stage a baseline on an ESXi host, and already fails there, then you know it is not getting its packages on the ESXi host. This is a good indication that there is most likely not enough space. This can also happen in the remediation step, which I presume, unpack those packages, which of course also need space.
Issue 2: No persistent storage, check out the scratch folder & downloads folder
I lot of customers don’t use persistent storage within their vSphere environment to store the scratch partition or for their sysdump location. Often, there is no scratch partition available or the downloads folder within that partition is not available. You can solve this by creating those folders within the /tmp folder. Which you can also read more about in my blogpost here
Issue 3: Blocking Vibs or too much updates at the same time in the baseline
This one is more complicated, but often when there is not enough space, too many updates at the same time can block the upgrade as well. This can have multiple reasons, like I said, not enough space to stage and remediate all the updates at the same time, or maybe there is a certain vib that has an underlying problem when it is trying to be applied. The last one is more difficult to find out, and shouldn’t happen actually, but it does.
A way to solve this, is by simplifying your baseline. Try not to update to many vibs at the same time, but start with the latest patch release. Then afterwards, check again if there are still vibs that need to be updated. It is cumbersome, but often this solves your issue.
Btw. From what I’ve seen in my tests, the patches are cumulative. I know a Major update is cumulative to the previous patches, but apparently the latest minor patches should also be cumulative. Still it is not a bad practice to always check double if this is the case. Sometimes this is not. I recently was also in the preasumption that my latest patch had the VMware Tools fix for windows machines for one of the VMware Security Advisory (VSA) issues that came out. However this was not the case. So always check.
Also it is also possible that you have a vib that is just blocking the upgrade. This should be seen in the logs of the ESXi host
Some other tips
If the above doesn’t help, there are still a few things that you could do.
- Check the vmfs storage once more, delete all the unnecessary stuff in the tmp folder or the vmfs volume that is using up space, and then give it a reboot. For one of my ESXi host, this was the magic combo that made it finally push the update.
- Check log files, it is troublesome, but you can often find most issues there after searching for a while. Especially, when you have blocking vibs. (/var/logs)
Work arounds
If al is just doomed to work, you can still workaround VUM the following:
- Reinstall the ESXi host, and reapply the profile and vib. I know, it is a little extreme but it works often.
- Update the ESXi host with the ISO trhough ILO, IDRAC, CISCO CIM, et cetera. This almost never fails, but the problem is, minor releases don’t often have a separate ISO for VMware. So you have create it yourself or your vendor should have a Custom ISO with the latest patches in it.
- Download the offline patch, and install the vibs yourself. Patches can be found here:
https://my.vmware.com/patch/#search
Hope that this was helpful
Samir is the author of vSAM.Pro & a Life enthusiast who works as a consultant in the field of IT. With a great passion for Tech & Personal Development, he loves to help people with their problems, but also inspire them with a positive outlook on life.
Besides that, he is also a big Sport & Music junky that loves to spend a big chunk of his time on producing music or physically stretching himself.
