This article is intended to help users troubleshoot operating system hangs on hosts with Parallels Virtuozzo Containers and Parallels Server software installed.
It describes general practices of issue localization and distinguishing between operating system hangs, resource shortages, and other component (GUI, networking, etc.) slowdowns.
Diagnose the issue
Check which part of the system is slow or unresponsive:
Graphical interface - Cursor or keyboard commands are delayed; elements of GUI are not manageable.
In such cases, the slowdown may be caused by a weak graphical subsystem configuration or a slow network connection.
To get rid of the issue, it is necessary to set GUI settings to lower values. If that does not help, read on.
Networking - Commands are executed quickly on the server, but the output is slow or delayed.
Usually, network performance is limited by the performance of the network infrastructure or its configuration.
It is worth checking the network configuration to avoid IP or MAC address duplicates.
Incorrect bonding configuration may cause significant network performance degradation, so make sure that the network equipment supports interface teaming.
In cases of intensive network activity on the server, it is useful to increase TCP memory limits: How to tune network performance on PVC and PSBM
Virtual machine or container processes - Only a specific virtual machine or container exhibits bad performance, while other virtual environments are running fine.
Most likely, container or virtual machine resources have been exceeded, and this is causing slow performance. It is useful to check failed UBC counters to find the exceeded limits: What are User Beancounters?
Also, it is useful to monitor resources of containers that work in SLM mode: How to continuously capture a Linux host node and Containers utilization (SLM, UBC)
If there are a lot of failed counters, it is advised to either decrease the allowed memory for allocation in the configuration of the most resource-consuming services (Apache, MySQL, etc.) or to increase the corresponding limits for the container in question.
If the virtual machine seems to be frozen, it is worth checking the performance counters to see if there is any activity: Monitoring virtual machine state with "prl_perf_ctl"
If counters are changed while monitoring them in a loop, then the virtual machine process is not frozen and one resource-consuming application is probably taking up all of the resources of the virtual machine. In such cases, increasing memory limits for the virtual machine may help.
The server itself - Processes are not getting queued for execution fast enough; the server shows a big load average and responses to the local keyboard are slow or delayed.
First, it is necessary to check if there is memory oversell or not. In other words, the total sum of memory allocated for containers and virtual machines on the server should be less than RAM + SWAP - 1GB.
Next, it is worth checking the speed of disk I\O operations. In the case of a slow hard disk, with read and write operations, it is possible to maximize RAM usage and minimize SWAP usage to increase server performance: How to tweak disk I\O performance on Linux hosts
Finally, it is suggested that you check the dmesg command output to find any hardware-related messages, such as I\O operation failure.
Prepare the node for troubleshooting:
Set up crash dump generation to be able to trigger a system crash manually:
- For Windows, check the article How to configure memory dump generation on Windows.
- For Linux, check the article How to configure kernel crash dumps on a Linux server.
Configure the serial console connection to be able to efficiently troubleshoot server freezes:
- For Windows, use the article How to set up a serial console to a Windows server.
- For Linux or Bare Metal, use the article How to set up a serial console to a Linux server.
Collect all necessary information
Parallels Support needs the following information to troubleshoot these types of issues:
All relevant logs can be submitted via built-in tools:
On a Parallels Virtuozzo Containers server, send a "VZReport":
How do I submit a Support Issue?
- On Parallels Cloud Server and Parallels Server Bare Metal, send a Problem Report:
~# prlsrvctl problem-report --send
- Compressed memory dumps
- Access to the Hardware Node
- Access to the Monitoring Node
Submit support ticket
- If you do not have a support ticket open, please create one first.
- If you have the ticket, provide the issue description in that ticket.
- Log files are submitted via built-in tools as described above.
- Contact a Parallels Support representative with the support ticket and request the link to upload the dump to.
For more information about server crash scenarios, see this article: How to handle a server crash scenario
What are User Beancounters?
How to continuously capture a Linux host node and Containers utilization (SLM, UBC)
How to set up a serial console to a Windows server
How to set up a serial console to a Linux server
How to configure kernel crash dumps on a Linux server
Using ALT+SYSRQ Keyboard Sequences
How to tune network performance on PVC and PSBM
Monitoring virtual machine state with "prl_perf_ctl"
How to tweak disk I\O performance on Linux hosts