VM | Ward Vissers

How to Remove Inaccessible vSAN Objects in vSphere: Step-by-Step Guide

This post is about how to remove such an inaccessible object within vSAN.

Afbeelding met tekst, schermopname, nummer, Lettertype

Automatisch gegenereerde beschrijving

Open an SSH session to the vCenter and enter the command rvc localhost in the command line.

Navigate to the destinated vSAN cluster where you want to remove the inaccessible objects using cd and utilize ls to list in each step like this one:

Verify the state of vSAN objects using the command vsan.check_state -r . This check involves three steps:

Checking for inaccessible vSAN objects
Verifying invalid or inaccessible VMs, and
Checking for VMs for which VC/hostd/vmx are out of sync

During this check, as you can see in the next screenshot, there are four inaccessible objects with the same UUID as those listed in Virtual Objects within the vSphere Client.

Afbeelding met tekst, schermopname, Lettertype

Automatisch gegenereerde beschrijving

To remove them, open an SSH session to any ESXi in the cluster and use the following command /usr/lib/vmware/osfs/bin/objtool delete -u <UUID> -f replacing UUID with the one you want to remove. Afbeelding met tekst, schermopname, Lettertype

Automatisch gegenereerde beschrijving

After you remove all inaccessible objects and run the (vsan.checkstate -r .) once again, you should no longer see any inaccessible objects. Afbeelding met tekst, schermopname, software, Lettertype

Automatisch gegenereerde beschrijving

‘Ineligible for use by VSAN’ can’t be added to VSAN disk groups

I had the opportunity to test a Dell vSAN node. I had a older unattend install esxi iso.
This installed the ESXi OS on the wrong disk. After a correct install vSAN did not see this this disk ready for use for vSAN. Combining the following articles Dell VXRai vSAN Drives ineligible and identify-and-solve-ineligible-disk-problems-in-virtual-san/
I solved this problem with the following steps:

Step 1: Identify the Disk with vdq -qH

Step 2: Use partedUtil get “/dev/disks/<DISK>” to list all partitions:

partedUtil get “/dev/disks/t10.NVMe____Dell_Ent_NVMe_CM6_MU_3.2TB______________017D7D23E28EE38C”

Step 3: Use This disk has 2 partitions. Use the partedUtil delete “/dev/disks/<DISK>” <PARTITION> command to delete all partitions:

Afbeelding met tekst, Lettertype, schermopname

Automatisch gegenereerde beschrijving

Step 4:

When all partitions are removed, do a rescan:

~ # esxcli storage core adapter rescan –all

Afbeelding met tekst, schermopname, software, Computerpictogram

Automatisch gegenereerde beschrijving

Step 5: Claim Unused Disks

Afbeelding met tekst, software, Lettertype, nummer

Automatisch gegenereerde beschrijving

ESXi Unattend Install on Dell BOSS controller

I had the opportunity to test a Dell vSAN node. I had a older unattend install esxi iso.
This installed the ESXi OS on the wrong disk.

I hate to type a very complex password twice.
So automation is the key.
I love de ks.cfg install option

Sow following the following guide did not the trik:
https://www.dell.com/support/kbdoc/en-us/000177584/automating-operating-system-deployment-to-dell-boss-techniques-for-different-operating-systems

VMware ESXi Automated Install

This did not work:
install –overwritevmfs –firstdisk=”DELLBOSS VD”

After doing a manual install:
Afbeelding met tekst, schermopname

Automatisch gegenereerde beschrijving

What works:

# For Dell Boss Controller “Dell BOSS-N1″

install –overwritevmfs –firstdisk=”Dell BOSS-N1”

My HomeLab anno 2024

My home lab is manly used for testing new stuff released by VMware (70%) en Microsoft (20%) other stuff (10%)

For the Base I use my Home PC

Intel i5 12600k
128 GB Memory
2 x 2TB Samsung 980 and 990 Pro SSD.
Windows 11 Pro
VMware Workstation Pro

On my Home PC running
Server 2022 (Eval for DC)
ESXi801 (16 GB) (NSX Demo Cluster)
ESXi802 (16 GB) (NSX Demo Cluster)
ESXi803 (64 GB) (General Cluster) )
ESXi804 (64 GB) (General Cluster)
ESXi805 (24 GB) (Single Node vSAN Cluster)
ESXi806 (16 GB) (4 Node vSAN Cluster)
ESXi807 (16 GB) (4 Node vSAN Cluster)
ESXi808 (16 GB) (4 Node vSAN Cluster)
ESXi809 (16 GB) (4 Node vSAN Cluster)

ESXi701 (24GB) (General Cluster)
ESXi702 (24GB) (General Cluster)

In general cluster there a running the most VM’s. Also here I am testing Packer and Terraform.

Afbeelding met tekst, schermopname, software, Lettertype

Automatisch gegenereerde beschrijving

For a while I used a 2TB Samsung SSD a Storage for ESXi Server through Truenas
But I wanted a larger storage for all my VM’s.

After reading on William Liam blog Synology DS723+ in Homelab and Synology NFS VAAI Plug-in support for vSphere 8.0

So I did a nice upgrade. Afbeelding met tekst, schermopname, software, nummer

Automatisch gegenereerde beschrijving

I used not the original Synology Parts. Following parts works fine.
Kingston 16 GB DDR4-3200 notebook memory
WD Red SN700, 500 GB SSD
WD Red Pro, 8 TB

* For Read-Write caching you need 2 SSD devices.

For mouting the NFS shared I created a little powercli script.

https://github.com/WardVissers/VMware-Powercli-Public/blob/main/Add%20NFS%20DataStore%20Github.ps1

Windows Server 2025 “Preview” deployment with Packer

As Windows Server 2025 Preview is officially released, I wanted to test a automated build of the Windows Server 2025 Preview release. So that I can deploy this in my home lab and going to test the new features if I can find the time….

About Hashicorp Packer

Hashicorp Packer is a self-contained executable producing quick and easy operating system builds across multiple platforms. Using Packer and a couple of HCL2 files, you can quickly create fully automated template(s) with latest Windows Updates en VMware Tools. When you schedule a fresh builds after patch Tuesday you have always an up-to-date and fully secured template.

When using VMware customization tools. You can spin up vm’s in minutes.

Automated Windows Server 2025 “Preview” Build

Files you need?
The files and versions I am using at the time of this writing are as follows:

Outside of downloading both Packer and Windows Server 2022 Preview build, you will need the following files:

windowsserver2025.auto.pkrvars.hcl – houses the variable values you want to define.
windows2025.json.pkr.hcl – the Packer build file
Answer file – Generated with Windows System Image Manager (SIM) you can download the file below
Custom script file(s) – optional

Other considerations and tasks you will need to complete:

Copy the Windows Server 2025 ISO file to a vSphere datastore

Windows Server 2025 unattend Answer file for the automated Packer Build

Like other automated approaches to installing Windows Server, the automated Windows Server 2025 Packer build requires an answer file to provide answers to the GUI automatically and other installation prompts that you normally see in a manual installation of Windows Server.

You will find the scripts here: https://github.com/WardVissers/Packer-Win2025

The only problem that I had was: Switching from Nic from Public to Private

# Set network connections profile to Private mode.

Write-Output ‘Setting the network connection profiles to Private…’

do {

$connectionProfile = Get-NetConnectionProfile

Start-Sleep -Seconds 10

} while ($connectionProfile.Name -eq ‘Identifying…’)

Set-NetConnectionProfile -Name $connectionProfile.Name -NetworkCategory Private

Windows Server 2025 Preview (Build: Canary 26052)

I had some time to check out the new version of Server 2025.

For the full upcomming features check: https://ignite.microsoft.com/en-US/sessions/f3901190-1154-45e3-9726-d2498c26c2c9?source=sessions

Download Server 2025 Preview: https://www.microsoft.com/en-us/software-download/windowsinsiderpreviewserver

Server 2025 will come with a lot of features (My Top 20+):

General – Server 2022 upgrade to .vNext (Controled bij GPO)
Hot Patching (Arc Enabled, Monthly Subscription)
Active Directory – 32k page
Active Directory – Numa
Active Directory – LDAP TLS 1.3
Active Directory – Improved Security for Confidential Attributes
Active Directory – Active Directory LDAP prefers Encryption bij Default
Active Directory – Kerberos Support for AES/SHA256/384
Active Directory – Changes to Default behavior of legacy SAM RPC Spassword change methods
Active Directory – Kerberos en KPINT Support cryptographic agility
Active Directory – New AD Forest en Domein Level (Minimal Server 2016 requirement)
Storage – NVME 70%/90% peformance increase
File Server – SMB over Internet (Quick Protocol)
File Server – More Control over SLTM
File Server – SMB Limitor (Enabeld bij Default)
File Server – Signing by Default
File Server – Minimum version SMB
File Server – More Secure Bij Default (Netbios disabled bij default)
RDS – M365 Apps stil supported for every Windows Server release 2-3 years
Finance – General support and Pay-as-you-go Support

Need to find some time to dig in

Handy link: https://techcommunity.microsoft.com/t5/windows-server-insiders/announcing-windows-server-preview-build-26040/m-p/4040858

VCF 5.0 running inside Nested ESXi server with only 64GB Memory

So I interested to trying to deploy latest release of VMware Cloud Foundation (VCF) 5.0 on my Windows 11 Home PC witch have 128GB and 16 core intel cpu.

William Lee wrote a nice artikel about VMware Cloud Foundation 5.0 running on Intel NUC

Disclaimer: This is not officially supported by VMware, please use at your own risk.

Requirements:

VMware Cloud Builder 5.0 OVA (Build 21822418)
VCF 5.0 Licenses Through VMUG ADVANTAGE
Home PC (Not Special Hardware)
– 128GB Memory
– Intel 12600 CPU
– 4TB of NVME Storage
Windows 11 with VMware Workstation 17

Setup

Virtual Machines

DC02 (Domain Controller, DNS Server) (4GB 2vcpu)
VCF-M01-ESX01 (ESXi 8.0 Update 1a) (64GBGB 1x140GB 2x600NVME 2x NIC) (Every Thin Provisiond)
VCF-M01-CB01 (4GB and 4CPU) Only needed through First Deploment

Network settings on my PC

1 IP In my home network
172.16.12.1 (To Fool Cloudbuilder)
172.16.13.1 (To Fool Cloudbuilder)

Procedure:

Install en Configure ESXi

Step 1 – Boot up the ESXi installer from de iso mount and then perform a standard ESXi installation.

Step 2 – Once ESXi is up and running, you will need to minimally configure networking along with an FQDN (ensure proper DNS resolution), NTP and specify which SSD should be used for the vSAN capacity drive. You can use the DCUI to setup the initial networking but recommend switching to ESXi Shell afterwards and finish the require preparations steps as demonstrated in the following ESXCLI commands:

esxcli system ntp set -e true -s pool.ntp.org
esxcli system hostname set –fqdn vcf-m01-esx01.wardvissers.nl

Note: Use vdq -q command to query for the available disks for use with vSAN and ensure there are no partitions residing on the 600GB disks.
Don’t change time server pool.ntp.org.

To ensure that the self-signed TLS certificate that ESXi generates matches that of the FQDN that you had configured, we will need to regenerate the certificate and restart hostd for the changes to go into effect by running the following commands within ESXi Shell:

/bin/generate-certificates
/etc/init.d/hostd restart

Cloudbuilder Config

Step 3 – Deploy the VMware Cloud builder in a separate environment and wait for it to be accessible over the browser. Once CB is online, download the setup_vmware_cloud_builder_for_one_node_management_domain.sh setup script and transfer that to the CB system using the admin user account (root is disabled by default).

Step 4 – Switch to the root user and set the script to have the executable permission and run the script as shown below

su –
chmod +x setup_vmware_cloud_builder_for_one_node_management_domain.sh
./setup_vmware_cloud_builder_for_one_node_management_domain.sh

The script will take some time, especially as it converts the NSX OVA->OVF->OVA and if everything was configured successfully, you should see the same output as the screenshot above.

A screenshot of a computer

Description automatically generated

Step 4 – Download the example JSON deployment file vcf50-management-domain-example.json and and adjust the values based on your environment. In addition to changing the hostname/IP Addresses you will also need to replace all the FILL_ME_IN_VCF_*_LICENSE_KEY with valid VCF 5.0 license keys.

Step 5 – The VMnic in the Cloud Builder VM will acked als a 10GB NIC so I started the deployment not through powershell but normal way in Cloud Builder GUI.

Your deployment time will vary based on your physical resources but it should eventually complete with everything show success as shown in the screenshot below. (I have one retry for finish)
A screenshot of a computer

Description automatically generated A screenshot of a cloud support

Description automatically generated
Here are some screenshots VCF 5.0 deployment running on my home PC.

A screenshot of a computer

Description automatically generated

Problems

Check this if you have problems logging in NSX:
https://www.wardvissers.nl/2023/07/26/nsx-endless-spinning-blue-cirle-after-login/

Next Steps.

1. Reploy with use of the Holo-Router https://core.vmware.com/resource/holo-toolkit-20-deploy-router#deploy-holo-router

2. Testing if can deploy Single Host VCF Workload Domain, on same way by following this blog post HERE! 😁
A screenshot of a computer

Description automatically generated

If I can start another 64GB ESXi Server.

Powercli Delete Orphaned Edge Transport Nodes

I made a little mistake on my Lab Environment. There fore it was required to clean up my last unremoveable edge server.

Sow i made a little powercli script

RemoveOrphanedEdge.ps1

# https://kb.vmware.com/s/article/892835
# https://virtuallyjason.blogspot.com/2020/11/using-nsx-t-api.html

$nsxfqdn = Read-Host “Give the IP of the FQDN from your NSX Server”
if (!($creds)){

$creds = Get-Credential -Message "NSX-T Administrative Credentials:"

}
$base64Creds = [Convert]::toBase64String([System.Text.Encoding]::UTF8.GetBytes(“$($creds.username):$($creds.GetNetworkCredential().password)”))
$header = @{Authorization = “Basic $base64Creds”}
$uri = “https://$nsxserver”

# Check API Connection

invoke-webrequest -uri “$uri/api/v1/spec/openapi/nsx_api.json” -headers $header -SkipCertificateCheck

# Clean Stale Entries

invoke-webrequest -uri “$uri/api/v1/transport-nodes?action=clean_stale_entries” -headers $header -Method ‘POST’ -SkipCertificateCheck

Tag: VM

How to Remove Inaccessible vSAN Objects in vSphere: Step-by-Step Guide

Like this:

ESXi Unattend Install on Dell BOSS controller

Like this:

My HomeLab anno 2024

Like this:

Windows Server 2025 “Preview” deployment with Packer

About Hashicorp Packer

Automated Windows Server 2025 “Preview” Build

Windows Server 2025 unattend Answer file for the automated Packer Build

Like this:

Windows Server 2025 Preview (Build: Canary 26052)

Like this:

Powercli Delete Orphaned Edge Transport Nodes

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

About Hashicorp Packer

Automated Windows Server 2025 “Preview” Build

Windows Server 2025 unattend Answer file for the automated Packer Build

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Setup

Virtual Machines

Network settings on my PC

Procedure:

Install en Configure ESXi

Cloudbuilder Config

Problems

Next Steps.

Share this:

Like this:

Share this:

Like this:

Share this:

Like this: