Adding Default Gateway in ESXI via CLI/SSH
To add default gateway in esxi server using CLI/SSH, login to ssh & issue below cmd’s
1 2 3 4 5 6 7 8 9 10 11 12 13 | #Check Current Gateway , if any#CMD 1esxcli network ip interface ipv4 get#CMD 2esxcli network ip interface list#CMD 3esxcfg-route>>> VMkernel default gateway is 0.0.0.0# Add Default Gateway 10.0.0.1 (change as per actual)esxcfg-route -a default 10.0.0.1>>> VMkernel default gateway set to 10.0.0.1 #Verify if route have been added. esxcfg-route >> VMkernel default gateway is 10.0.0.1 |
VCSA 6.7 | 503 errors while accessing vcsa portal
503 Error- Solution # 1
While accessing VCSA via web browser, we faced following error
Solution#1:
Login to VCSA by SSH, & issue below cmd’s
1 2 3 | shell.set --enabled trueshellservice-control --status --all |
if you see services in STOPPED: section, then issue below cmd to start all services
1 | service-control --start --all |
Now try to access the VCSA via browser & it should work fine.
503 Error- Solution # 2
After logging to VCSA via SSH , and issue df -h, you see SEAT partition full or if you login to the management panel you see following,
If you are seeing seat partition getting near full, its better to extend that disk via ESXI server & then extend it using VCSA cmd’s. Also you can truncate the DB to remove old data logs.
Solution#1:
Login to VCSA by SSH, & issue below cmd’s
1 2 3 4 | shell.set --enabled trueshell# Login to Database/opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres |
This will display the top 30 largest tables within the vCenter Server database
1 | SELECT nspname || '.' || relname AS "relation", pg_size_pretty(pg_total_relation_size(C.oid)) AS "total_size" FROM pg_class C LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace) WHERE nspname NOT IN ('pg_catalog', 'information_schema') AND C.relkind <> 'i' AND nspname !~ '^pg_toast' ORDER BY pg_total_relation_size(C.oid) DESC LIMIT 30; |
You will see tables as per following
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | VCDB=# SELECT nspname || '.' || relname AS "relation", pg_size_pretty(pg_total_relation_size(C.oid)) AS "total_size" FROM pg_class C LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace) WHERE nspname NOT IN ('pg_catalog', 'information_schema') AND C.relkind <> 'i' AND nspname !~ '^pg_toast' ORDER BY pg_total_relation_size(C.oid) DESC LIMIT 30;relation | total_size---------------------+------------vc.vpx_event_arg_8 | 145 MBvc.vpx_event_arg_2 | 143 MBvc.vpx_event_arg_1 | 143 MBvc.vpx_event_arg_92 | 143 MBvc.vpx_event_arg_10 | 143 MBvc.vpx_event_arg_11 | 143 MBvc.vpx_event_arg_16 | 143 MBvc.vpx_event_arg_13 | 142 MBvc.vpx_event_arg_3 | 142 MBvc.vpx_event_arg_90 | 142 MBvc.vpx_event_arg_9 | 142 MBvc.vpx_event_arg_4 | 142 MBvc.vpx_event_arg_91 | 142 MBvc.vpx_event_arg_15 | 142 MBvc.vpx_event_arg_14 | 142 MBvc.vpx_event_arg_87 | 142 MBvc.vpx_event_arg_5 | 142 MBvc.vpx_event_arg_12 | 142 MBvc.vpx_event_arg_88 | 142 MBvc.vpx_event_arg_89 | 142 MBvc.vpx_event_arg_6 | 142 MBvc.vpx_event_arg_21 | 142 MBvc.vpx_event_arg_22 | 141 MBvc.vpx_event_arg_23 | 141 MBvc.vpx_event_arg_17 | 141 MBvc.vpx_event_arg_7 | 141 MBvc.vpx_event_arg_19 | 140 MBvc.vpx_event_arg_24 | 140 MBvc.vpx_event_arg_18 | 140 MBvc.vpx_event_arg_20 | 140 MB(30 rows) |
Now by seeing the table names, you can Truncate the large tables individually
For example:
1 | truncate table vc.vpx_event_arg_8 cascade; |
Keep cleaning as per disk space requirements, then Exit the vCenter Server Appliance database by running this command:
\q
Verify the space is reclaimed by running the df -h command.
Preferably restart the VCSA services or restart the VCSA server so that alerts can clear up quickly
Read below vmware guide for more information
VCSA DB Tables Automatic cleanup via SCRIPTING ! Workaround using BASH SCRIPTING
First, Connect to VCSA via putty, & set shell
1 2 | shell.set --enabled trueshell |
Now create two files, one is the bash file that will execute an sql query, & then the second file which will hold the sql query.
First the SQL query file
1 | vi /root/dbcleanup.sql |
& paste following data in it.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | DO$$DECLARErec record;BEGINFOR rec INSELECT *FROM pg_tablesWHERE tablename ~ '^vpx_event_[0-9].*'ORDER BY tablenameLOOPEXECUTE 'TRUNCATE TABLE '|| quote_ident(rec.schemaname) || '.'|| quote_ident(rec.tablename) || ' CASCADE';END LOOP;END$$;DO$$DECLARErec record;BEGINFOR rec INSELECT *FROM pg_tablesWHERE tablename ~ '^vpx_event_arg_[0-9].*'ORDER BY tablenameLOOPEXECUTE 'TRUNCATE TABLE '|| quote_ident(rec.schemaname) || '.'|| quote_ident(rec.tablename) || ' CASCADE';END LOOP;END$$; |
Now the BASH script that will actually trigger the above dbcleanup.sql query file
1 | vi /root/dbcleanup.sh |
and paste following lines in it
1 2 3 4 5 6 7 8 9 | #!/bin/bash#> /var/mail/rootDT=`date`echo "$DT - DB process started ..." >> /var/log/db.log# to get the PGPASSWORD, pull it from this cmd > "cat /etc/vmware-vpx/embedded_db.cfg"export PGPASSWORD='w@x!8q6FB1HvmU9='/opt/vmware/vpostgres/current/bin/psql -U vc -d VCDB postgres -f /root/dbcleanup.sql -h /var/run/vpostgresDT=`date`echo "$DT - DB process ended ..." >> /var/log/db.log |
(-h /var/run/vpostgres syntax was necessary to execute the psql CMD, it took 1 day to resolve, without this, you will get psql connect error)
Press :wq to save & exit VI editor.
Now schedule it to run every hour or as you like
1 | crontab -e |
Press i to go in edit mode and paste following
1 2 3 4 | # to execute the table cleanup script for every 5 minute,*/5 * * * * /root/dbcleanup.sh# to execute the table cleanup script every HOUR,@hourly /root/dbcleanup.sh |
Now press Escape, and then :wq which will result in save & exit the vi editor.
Congrats ! all done.
This was written in particular to VCSA ver 6.7 only !
Syed.Jahanzaib
ESXI Server: ‘SEL Fullness’ message
Under ESXI, or VCENTER , I was seeing following alert ..
System Management Software 1 SEL Fullness: Log almost full
Solution # 1
In vSphere 6.7 , Navigate to following and reset event log
Monitor > Tasks and Events > Hardware Health > SYSTEM EVENT LOG > RESET EVENT LOG
Solution # 2
Enable SSH on that esxi host, and connect to it via ssh client & issue below cmd’s
1 2 3 | localcli hardware ipmi sel clear/etc/init.d/sfcbd-watchdog restartservices.sh restart |
For few minutes it will show disconnected in vcenter, after few minutes, Refresh the Webclient & related errors will be cleared.
Alhamdolillah!
Updating ESXI 6.5 to 6.7u3 from the CLI
Recently I did some major updates in our virtual infrastructure including ESXi (ver 6.5.0, 10719125) , Vcenter & Veeam (9.5.4.2866) . Things were updated in following order
- Veeam B&R upgraded to ver 9.5.4.2866
- Vcenter upgraded to ver 6.7.0.40000
- ESXI hosts upgraded to ver 6.5.0, 10719125
For ESXI update from 6.5 to 6.7u3. In the past I always use Installer CD/USB to upgrade from older esxi to new version, but for this approach, I have to compromise on my holidays or sit very late in office. This time time I took another approach and upgraded all the esxi hosts one by one on sundays remotely from the home using offline bundle installer & esxi CLI method. This is how I accomplished it.
I first downloaded the 6.7 update offline bundle from the Lenovo site (since we have all the IBM/Lenovo brand servers therefore I selected this option to avoid any hardware driver issue).
Afterwards I uploaded this offline bundle zip file to Esxi datastore, then logged in to esxi host via SSH, and issued
1 | esxcli software vib install -d /vmfs/volumes/5d0cf64f-a83e7c86-6a4d-40f2e922c64a/Lenovo_Offline_Bundle_VMware_ESXi_6.7.0.update03_14320388_LNV_20190920.zip |
Note: make sure to change datastore and filename as required.
It took few minutes, once I saw SUCCESS message, I completed the process by simply rebooted the ESXI host by cmd
1 | reboot |
Better approach is to update rather than install
1 | esxcli software vib update -d /vmfs/volumes/5d0cf64f-a83e7c86-6a4d-40f2e922c64a/Lenovo_Offline_Bundle_VMware_ESXi_6.7.0.update03_14320388_LNV_20190920.zip |
Difference between VIB update and VIB install
Excerpt from “https://communities.vmware.com/thread/435959”
To install or update a .zip file, use the -d option. To install or update a .vib file use the -v option.
Using the update command is the recommended method for patch application. Using this command applies all of the newer contents in a patch, including all security fixes. Contents of the patch that are a lower revision than the existing packages on the system are not applied.
Using the install command overwrites the existing packages in the system with contents of the patch you are installing, including installing new packages and removing old packages. The install command may downgrade packages on the system and should be used with caution. If required, the install command can be used to downgrade a system (only for image profiles) when the –allow-downgrade flag is set.
The install method has the possibility of overwriting existing drivers. If you are using 3rd party ESXi images, VMware recommends using the update method to prevent an unbootable state.
Check esxi version from CLI
1 2 3 4 5 6 | esxcli system version getProduct : VMware ESXiVersion : 6.7.0Build : Releasebuild-14320388Update : 3Patch : 73 |
