Popular Posts

Wednesday, November 4, 2026

Ubuntu Server 26.04 LTS - Setting up GUI and fixing network issues

Background

You have spin up an Ubuntu Server instance running version 26.04 TLS and you want to install GUI on your server.


Steps to install GUI and its related components

  1. Run sudo apt update to update package repositories

  2. Run sudo apt install ubuntu-desktop to install GNOME desktop environment. This includes GUI + gnome-remote-desktop + gdm3
    This version of Ubuntu Server includes GNOME 50, which no longer supports X11 sessions. So, it is not possible to couple GNOME desktop with other RDP server such as XRDP. Ensure only one of them is running as solutions operating at the same will result in error messages such as gnome-session-init-worker crashed with SIGABRT in _libc_start_call_main()



  3. Run sudo apt install gdm3 to install display manager if not installed already by ubuntu-desktop
    You may also install other display managers such as lightdm or kdm (now called SDDM) in place of gdm3. As with RDP service, ensure only one display managers is running if multiple display managers are installed. This can be verified by running cat /etc/X11/default-display-manager . Expected output is /usr/sbin/gdm3 for gdm3

     

  4. Run sudo systemctl set-default graphical.target to boot into GUI mode by default

  5. Restart the server sudo reboot

  6. Once boot up, run sudo grdctl --system status to confirm if GNOME RDP is listening on port 3389

  7. Configure RDP by either:
    • Console
      1. Login to the GUI using console (ex: using VMRC on VMware)
      2. Open Settings → System → Remote Desktop
      3. Select Remote Login
      4. Click Unlock and authenticate
      5. Enable Remote Login
      6. Configure the RDP login credentials
    • SSH, run the following commands
      1. sudo grdctl --system rdp enable
      2. sudo grdctl --system rdp set-auth-methods credentials
        The credentials you configure here will be used to authenticate with the RDP server before entering credentials to a local account
      3. sudo grdctl --system rdp set-credentials


Possible network issues

Installing GNOME desktop on top of Ubuntu Server introduces a complication in terms of network configuration of the server. For example, you might notice that your server is no longer able to access the internet after GNOME desktop has been installed. Also, the server will take longer to boot because it waits on the network resolution task and eventually fails.

This is because GNOME desktop and Ubuntu Server use 2 different mechanisms to manage the network configuration, thus causing a conflict when both are running at the same time. Default network manager on Ubuntu Server is systemd-networkd while NetworkManager is default on desktop (GUI). To verfiy this, you may run
sudo systemctl is-enabled NetworkManager.service systemd-networkd.service

Since this server will be primarily managed and used via the desktop, disable systemd-networkd by running the command sudo systemctl disable systemd-networkd.service and update the network configuration via GUI. Perform a quick reboot to ensure changes are intact upon boot.

Monday, October 26, 2026

Updating Veeam I/O filters on VMware hosts

Background

You have installed Veeam I/O filters on your VMware ESXi hosts but they are out of date, i.e. they are not running the same version as your Veeam Backup and Replication (VBR) Server. You have been tasked to update your ESXi servers.


Issue

When you run the update of ESXi, you would get the following message:

''vSphere ESX Agent Manager' service reported issue: Agency ########-############ is unable to determine required solutions from its configured offline depots", which suggests that you need to update the I/O filter from Veeam.

When you try to update the I/O filter on VBR, you would get the following message:

"Host <your ESXi hostname>: The agent's workflow is blocked until it's required solutions are re-mediated externally in vSphere Lifecycle Manager"


Solution

Follow these steps to destroy the stale agency via the Managed Object Browser (MOB).

  1. Identify the Agency ID from the error message or the eam.log file (format: Agency:########-############).
  2. Navigate to https://<vCenter-FQDN>/eam/mob/?moid=<Agency-ID>.
  3. Log in with administrator credentials.
  4. Select DestroyAgency.
  5. Click Invoke Method.
  6. Retry the cluster remediation task in the vSphere Client.
Proceed to update the hosts via Lifecycle Manager and then update the I/O filters via Veeam



Saturday, October 24, 2026

How to setup Room Finder on Exchange

Room Finder is a feature in Microsoft Exchange Online and Outlook that helps users find and book available meeting rooms. It displays room availability, capacity, location, and other facilities, making it easier to select a suitable meeting space when scheduling meetings. Behind the scenes, Room Finder represents collection of rooms grouped together by Exchange Distribution Groups. This is however more suitable and useful for rooms spread across several cities.

Imagine you need to add the following rooms into Room Finder:

  • City 1
    • Building 1
      • Room 1
      • Room 2
    • Building 2
      • Room 3
      • Room 4
  • City 2
    • Building 1
      • Floor 1
        • Room 5
        • Room 6
      • Floor 2
        • Room 7
  • City 3
    • Building 1
      • Room 8


Steps to configure Room Finder in your environment

  1. List down all existing Room Resources in your environment and group them by City, Building and Floor as above
  2. Go to Exchange Admin Console (EAC) --> Room Resource and update the City field of all the rooms your want to add into Room Finder. You may use Set-Place command for this purpose.
  3. Additionally, you may also set the floor parameter for your rooms using Set-Place command. Ex: Set-Place room1@yourdomain.com -Floor
  4. Run Connect-ExchangeOnline (Run this beforehand if you intend to use Set-Place for previous steps)
  5. To group rooms from Building 1 located in City 1,  run New-DistributionGroup -Name "City 1 Rooms - Building 1" -RoomList
  6. Take note of the UPN (User Principal Name) of the distribution group created above
  7. To add a room to the above, run Add-DistributionGroupMember -Identity city1rooms-building1@yourdomain.com -Member room1@yourdomain.com
  8. Continue to add more rooms as required ex: Add-DistributionGroupMember -Identity city1rooms-building1@yourdomain.com -Member room2@yourdomain.com
  9. Create another group and add rooms into it ex: New-DistributionGroup -Name "City 1 Rooms - Building 2" -RoomList
  10. Run Get-DistributionGroup -RecipientTypeDetails RoomList to verify the creation of those groups
  11. It may take up to 2 days for the Room List to update on Outlook client 
  12. To summarise
    1. You set the City field using EAC or PowerShell, this is compulsory.
    2. Group them into Distribution Groups by City and Building using PowerShell
    3. You may add other details such as the floor and capacity by updating the individual room resource. This is optional.
    4. Users will be able to filter rooms by city and building and additionally by floor and capacity if configured.

Saturday, April 18, 2026

Windows 11 April 2026 updates - BootDevice Not Found

Recent Windows Updates broke some Windows 11 devices, preventing them from booting.


Background

The affected devices will showcase the following symptoms:

  1. User initiates the updating process
  2. At some point, a message appears on the screen 'Something didn't go as planned. No need to worry-undoing changes. Please keep your computer on'.
  3. Device shutsdown/reboot
  4. Upon reboot, the following message appears 'BootDevice Not Found, Please install an operating system on your hard disk. Hard Disk - (3F0)'
  5. Device will continue to load into System Diagnostics page in BIOS

Solution

 

1. Boot to a Windows USB
2. Go through the prompts and select ‘Repair this PC’
3. Choose language
4. Select Troubleshoot -> Command Prompt
5. Enter BitLocker Recovery Key (if enabled)
6. Enter the following commands
1. diskpart
2. list disk
3. sel disk 0 (usually its 0, otherwise check which disk contains Windows from the previous command)
4. list vol
5. sel vol 1 (usually its 1, verify from the list vol command which volume is about 100MB and FAT32 format)
6. assign letter=Y (or any letter)
7. exit
8. bcdboot C:\Windows /s Y: /f UEFI
7. Close the command prompt
8. Select Exit and continue to Windows 11
9. Enter BitLocker Recovery Key (if enabled)
10. If Windows still doesn’t boot, repeat the above steps once more


Thursday, November 14, 2024

Setting up Ansible control node for managing Linux and Windows hosts

Background

In this post, I will walk you through the complete end-to-end procedure for setting up an Ansible controller node and setting up the connection for managing RHEL and Windows Server nodes.


Azure setup

I have created 3 servers and placed them in a simple network configuration.

Infra:

    10.0.3.4 ansiblecontrol (RHEL 8.7)

App:

    10.0.5.4 linuxnode1 (RHEL 8.7)

    10.0.5.5 windowsnode1 (Windows Server 2019)





OS preparation

Ensure all hosts are configured with the correct FQDN and IP configurations. This is more so important when dealing with domain-joined nodes.

ex:

  • run hostnamectl set-hostname ansiblecontrol.local.vanguard.com to set the hostname for RHEL
  • add DNS suffix to computer name on System Settings on Windows Server

Make sure the control node can reach the managed nodes in the absence of DNS by adding entries in /etc/hosts




Setting up Ansible on Control node

  1. Install ansible sudo yum install ansible

     2. Generate SSH key pair for ansible user ssh-keygen


     3. Copy public key to linuxnode1 ssh-copy-id linuxnode1.local.vanguard.com


     4. Install pywinrm for managing Windows nodes pip3 install "pywinrm>=0.3.0"



     5. Import windows module collection ansible-galaxy collection install ansible.windows




Setting up Linux managed node

    1. Add ansible user on linuxnode1 sudo useradd ansible
    2. Add ansible user in the sudoers file for password-less authentication for all elevated commands using sudo visudo




    Setting up Windows managed node

         1. Ensure PowerShell version is 5.0 and above using $PSVersionTable command. Also make sure .NET version is later than 4.0. However, with newer Windows Servers, this shouldnt be an issue.

        2. Set up WinRM listener on port 5986 for HTTPS transport. This requires the creation of a self-signed certificate. Ansible has provided a script for this task which is available to download on GitHub



        2. Additionally, make sure Windows Firewall is switched off


    Sample Ansible directory setup

    1. Create inventory file by grouping the 2 managed nodes by operating system


        2. Define group_vars.

            Create a new file called windows.yml in group_vars. The contents of the file defines the connection parameters that will be used by ansible to connect with hosts under the windows group of the inventory file. This will also work for any dynamic inventory as long as those hosts are also added to the windows group during playbook runtime.
             
            For domain-joined windows servers, it is advised to use the more secure kerberos instead of ntlm. In such cases, kerberos packages must be installed and krb5.conf configured on the ansible control node to obtain kerberos tickets from the AD domain.



        3. Encrypt windows group_vars using ansible-vault to protect sensitive information
            ansible-vault encrypt windows.yml
        

        4. Perform a ping to verify connectivity using ad-hoc ping command. Make sure to use --ask-vault-pass for windows nodes









    Sample playbook

     1. The following playbook will install webserver on both nodes depending on their OS



    2. Execute the playbook first by doing a dry-run

    ansible-playbook installwebserver.yml -i testservers --check --ask-vault-pass



    3. Run the playbook

    ansible-playbook installwebserver.yml -i testservers --ask-vault-pass





    Saturday, November 9, 2024

    Password sync issues between AD and Entra ID - ‘User must change password at next logon’

    Background

    You want to reset a user's password in an Entra ID-connected Active Directory environment.


    Issue

    In Active Directory Users and Computers, you reset a user's password by making sure 'User must change password at next login' is selected.









    The password was reset successfully. However, after allowing enough time for the Entra Connect sync, the user cannot authenticate with the new password on M365 and/or the company portal. The error message indicates an incorrect password.


    Troubleshooting steps

    In the Entra ID admin portal, the password for the same user is reset, enabling successful authentication into web-based services, with the updated password also reflected in Active Directory.

    It appeared that the password hash sync only works from Entra to AD, not the other way around.

    If the ‘User must change password at next logon’ flag was deselected when the password was reset, it would work across both Entra and AD.


    Conclusion

    The ‘User must change password at next logon’ flag will not sync with Entra unless the ForcePasswordChangeOnLogOn feature is enabled on the Entra ID tenant (see below).

    Implement password hash synchronization with Microsoft Entra Connect Sync - Microsoft Entra ID


    If the option ‘User must change password at next logon’ is selected in Active Directory, but that feature is not enabled in Entra ID, password changes will not be synced (see below).

    azure-content/articles/active-directory/active-directory-aadconnectsync-implement-password-synchronization.md at master · toddkitta/azure-content

    As a workaround, 'User must change password at next logon’ was deselected during password reset until changes were made to the Entra ID tenant.

    Friday, March 22, 2024

    How to change existing Azure VM from security type Trusted to Standard in order to enable nested virtualisation capabilities

    Background

    It is not possible to change the security type of an existing Azure VM back to standard. The only known way to achieve this is to setup a new standard VM and attaching the old OS disk to it.

    This process appears to be suitable for domain joined production environments as well, however, it may depend on the complexities of your own environment.


    Steps

    1. Stop the target VM.


    2. Export the current OS disk as VHD. Run the following script in cloud shell bash.

    #Provide the subscription Id where managed disk is created
    subscriptionId="your sub ID"

    #Provide the name of your resource group where managed disk is created
    resourceGroupName="rg name"

    #Provide the managed disk name
    diskName="current OS disk"

    #Provide Shared Access Signature (SAS) expiry duration in seconds e.g. 3600.
    #Know more about SAS here: https://docs.microsoft.com/en-us/azure/storage/storage-dotnet-shared-access-signature-part-1
    sasExpiryDuration=3600

    #Provide storage account name where you want to copy the underlying VHD file of the managed disk.
    storageAccountName="sa name"

    #Name of the storage container where the downloaded VHD will be stored
    storageContainerName="blob container for storing disks"

    #Provide the key of the storage account where you want to copy the VHD
    storageAccountKey="sa key"

    #Provide the name of the destination VHD file to which the VHD of the managed disk will be copied.
    destinationVHDFileName="name of VHD"

    az account set --subscription $subscriptionId

    sas=$(az disk grant-access --resource-group $resourceGroupName --name $diskName --duration-in-seconds $sasExpiryDuration --query [accessSas] -o tsv)

    az storage blob copy start --destination-blob $destinationVHDFileName --destination-container $storageContainerName --account-name $storageAccountName --account-key $storageAccountKey --source-uri $sas



    3. Create managed disk from the exported VHD by running the following script in PowerShell Cloud Shell.
        Make sure HyperVGeneration and Zone match the new VM that will be created in the next step.

    #Provide the subscription Id
    $subscriptionId = 'your sub ID'

    #Provide the name of your resource group
    $resourceGroupName ='rg name'

    #Provide the name of the Managed Disk
    $diskName = 'name of VHD'

    #Provide the size of the disks in GB. It should be greater than the VHD file size.
    $diskSize = '127'

    #Provide the URI of the VHD file that will be used to create Managed Disk.
    # VHD file can be deleted as soon as Managed Disk is created.
    # e.g. https://contosostorageaccount1.blob.core.windows.net/vhds/contoso-um-vm120170302230408.vhd
    $vhdUri = 'https://<your sa>.blob.core.windows.net/<container name>/VHDfilename.vhd'

    #Provide the resource Id of the storage account where VHD file is stored.
    #e.g. /subscriptions/6472s1g8-h217-446b-b509-314e17e1efb0/resourceGroups/MDDemo/providers/Microsoft.Storage/storageAccounts/contosostorageaccount
    $storageAccountId = '/subscriptions/<your sub id>/resourceGroups/<rg name>/providers/Microsoft.Storage/storageAccounts/<sa name>'

    #Provide the storage type for the Managed Disk. PremiumLRS or StandardLRS.
    $sku = 'StandardSSD_LRS'

    #Provide the Azure location (e.g. westus) where Managed Disk will be located.
    #The location should be same as the location of the storage account where VHD file is stored.
    #Get all the Azure location using command below:
    #Get-AzureRmLocation
    $location = 'your location'

    #Set the context to the subscription Id where Managed Disk will be created
    Set-AzContext -Subscription $subscriptionId

    #If you're creating an OS disk, add the following lines
    #Acceptable values are either Windows or Linux
    #$OSType = 'yourOSType'
    #Acceptable values are either V1 or V2
    #$HyperVGeneration = 'yourHyperVGen'

    #Specify Zone
    #Zone = 1

    #If you're creating an OS disk, add -HyperVGeneration and -OSType parameters
    $diskConfig = New-AzDiskConfig -SkuName $sku -Location $location -DiskSizeGB $diskSize -SourceUri $vhdUri -StorageAccountId $storageAccountId -Zone 2 -OsType Windows -HyperVGeneration "v2" -CreateOption Import

    #Create Managed disk
    New-AzDisk -DiskName $diskName -Disk $diskConfig -ResourceGroupName $resourceGroupName



    4. Delete current VM while retaining OS Disk (just in case). 
        Make sure new VM uses dsv4 series of Azure VMs that support nested virtualisation.



    5. Go to the new VM -> Disks, and choose Swap OS Disk. Choose the managed disk created in step 3.
        

    6. Re-assign IP on the new NIC on Azure


    7. Start new VM and connect using local administrator account
        After this step, OS disk provisioned along with new VM can be deleted.


    8. Remove old and hidden NICs in Device Manager (Select Show hidden devices under View menu)


    9. Edit Ethernet adapter by re-assigning IP along with default gateway and DNS server


    You should now be able to install Hyper-V role on your Azure VM





    Tuesday, August 1, 2023

    Adding domain CA trusted certificates to workgroup servers for SCCM connection

    Background

    In certain enterprise scenarios, it is possible to have isolated servers, a.k.a workgroup servers, that are not joined to the main Active Directory domain due reasons such as security requirements (ex: internet facing servers).

    Generally, Windows Server Clients should be part of the same domain as SCCM servers to be able to retrieve packages and security updates from SCCM Distribution Points (DPs). That is because SCCM needs to trust the client before it could push/pull packages and updates.

    However, there is a workround to this problem that involves installing domain certificates in workgroup servers that are not part of the domain. To add on, this could be automated using PowerShell.


    Steps

    1. From a domain connected client server, copy the following types of certs over to the target workgroup client server:

        a. Root

        b. CertificateAuthority

        c. TrustedPublisher

    2.  Run the following PowerShell script on the workgroup server with admin privileges. It uses .NET classes x509certificate2 and x509certificate. Make sure .NET 2.0 or above is installed on the target server.

    #define script log file
    $logfile = 'C:\Windows\Temp\Set-Certs.log'

    #function to write into the log file
    function writetolog([string] $txt) {
    $Stamp = (Get-Date).toString("yyyy/MM/dd HH:mm:ss")

    Add-content $logfile -value $Stamp':'$txt

    }

    #define list of certs
    $certs = @(@("<corp root>.cer","Root"),@("<CA authority 1>.cer","CertificateAuthority"),@("<CA authority 2, if any>.cer","CertificateAuthority"),@("<Trusted publisher>.cer","TrustedPublisher"))

    #define path to folder containing certs
    $certFolder = "C:\Users\admin\Desktop\WorkgroupCerts"

    #go through each cert
    foreach ($certName in $certs) {
    #start of the try/catch

    try {

    $cert = $certName[0]
    Write-Host $cert

    $storename = $certName[1]
    write-host $storename

    $certPath = "$certFolder\$cert"
    Write-Host $certPath

    #create a new certificate object using the x509certificate2 .NET class
    $newCert = new-object system.security.cryptography.x509certificates.x509certificate2

    #import the pfx file and use the x509KeyStorageFlag 'PersistKeySet' to set the certificate as persistent in our certificate store
    $newCert.Import($certPath)

    #get the thumbprint of the certificate going to be installed
    $newCertThumbprint = $newCert.Thumbprint

    #create a new object using the x509Store .NET class to manage the certificate catalog
    $store = New-Object System.Security.Cryptography.X509Certificates.x509Store("\\localhost\$storename", "LocalMachine")

    #open the certificate store with the 'ReadWrite' flag to be able to modify it
    $store.Open([System.Security.Cryptography.X509Certificates.OpenFlags].ReadWrite)

    #check if there is a certificate which match the new one's thumbprint present in the store
    $certPresent = Get-ChildItem -Path CERT:\\LocalMachine -Recurse | Where-object {$_.Thumbprint -eq $newCertThumbprint}

    #verify if the certificate is present. if not, install it
    if(!$certPresent) {

    Write-Host "Installing certificate on machine $machine"

    $store.add($newCert)

    }else{

    Write-Host "Certificate already present on machine $machine"

    }

    #close the store
    $store.Close()

    }

    catch {
    throw
    #update log
    Write-Host "The installation has Failed on the machine $machine"

    }


    }





    Monday, July 31, 2023

    Checking EMC Networker configuration on new Windows Server builds

    Background

    You have been tasked to find out if EMC Networker is configured properly on your new Windows Server builds. This could be achieved by running an Ansible Playbook coupled with a PowerShell script.


    Script

    Playbook

    #Define list of servers and their respective login details, this could be achieved differently depending on your infrastructure design
    - name: capture servers list
    hosts: localhost
    vars:
    - servers: "{{ myhost.split(',') }}"
    gather_facts: false
    tasks:
    - import_tasks: automation-add-group.yml

    - name: Check build status for windows
    hosts: temp_group
    gather_facts: false
    tasks:
    - name: Check for networker agent
    win_shell: if(Get-WmiObject -Class Win32_Product | where Name -eq 'Networker') { write-host "Found" } else { write-host "Not Found" }
    register: backupagent
    failed_when: "'Not Found' in backupagent.stdout"
    ignore_errors: true

    - name: make sure networker service is running
    win_service:
    name: nsrexecd
    state: started
    register: srtbckup
    ignore_errors: true

    #If networker service is not running
    - block:
    - name: rename nsr folder
    win_file:
    path: C:\Program Files\EMC NetWorker\nsr\res\nsrladb
    state: absent
    - name: make sure network service is running
    win_service:
    name: nsrexecd
    state: started
    when: srtbckup is failed
    ignore_errors: true

    - pause:
    seconds: 30

        #Calls the PowerShell script
    - name: Check backup configuration
    script: backservercheck.ps1
    register: backupserver
    failed_when: "'Backup client configured properly' not in backupserver.stdout"
    ignore_errors: true


    PowerShell

    In this example, we wanted to check the backup configuration on 2 sites where each site had their own Backup servers. Hence, this script taps on the naming convention of each Windows Server to determine which Backup server it should connect to.

    $hostname = $env:COMPUTERNAME

    if ($hostname -like '*naming convention 1*') {

    $backserverlist = @('backup server 1 - site 1','backup server 2 - site 1')

    foreach ($bckupserver in $backserverlist) {

     #Define nsr command
            $command = "echo print type: nsr client; name: $hostname | nsradmin -s $bckupserver -p nsrd"

     #Run nsr command in cmd
            $output = cmd.exe /c $command

    if($output -like '*scheduled backup: Enabled*') {

    Write-Host "Backup client configured properly"
    break

    }

    }

    } elseif ($hostname -like '*naming convention 2*') {

    $backserverlist = @('backup server 1 - site 2','backup server 2 - site 2')

    foreach ($bckupserver in $backserverlist) {

    $command = "echo print type: nsr client; name: $hostname | nsradmin -s $bckupserver -p nsrd"

    $output = cmd.exe /c $command

    if($output -like '*scheduled backup: Enabled*') {

    Write-Host "Backup client configured properly"
    break

    }

    }

    }

    Saturday, July 1, 2023

    Powershell script for checking Windows Server cluster health

    Background

    You have been tasked to write a script to check the health of your Windows Server clusters. Additionally, you are also required to email the cluster health report to your team.


    Script

    In order for the following script to work, FailoverClusters module must be imported to the sever that this script will run on. This can be achived by running the following command and it is a one-time task.

    Import-Module FailoverClusters


    param(

    #parameter 1, cluster server list
    [String]$clustersfile,
    #parameter 2, email the report or not. Default is Yes.
    [String]$emailtoteam="Yes"

    )

    #start of summary text
    $summary = "############## Summary ##############`r`n"

    #define log file
    $logfile = "path to log file_$(get-date -f yyyyMMddhhmmss).txt"

    $clusterlist = get-content $clustersfile

    foreach ($cluster in $clusterlist) {

    $errorflag = 0

    Write-Host $cluster

    $summary += $cluster

    $cluster | Out-File $logfile -Append

    if(Get-Cluster -Name $cluster -ErrorAction Continue) {

    ### cluster node
    $clusternodes = Get-ClusterNode -Cluster $cluster
    $clusternodes | ft | Out-File $logfile -Append

    foreach ($clusternode in $clusternodes) {

    if(($clusternode | select -ExpandProperty State) -ne "Up") {

    $errorflag += 1 #if the status is not UP, then report error

    }

    }

    ### cluster resource
    $clusterresources = Get-ClusterResource -Cluster $cluster
    $clusterresources | ft | Out-File $logfile -Append

    foreach ($clusterresource in $clusterresources) {

    if(($clusterresource | select -ExpandProperty State) -ne "Online") {

    $errorflag += 1 #if the status is not ONLINE, then report error

    }

    }

    if($errorflag -ne 0) {

    "ERROR: Check cluster`r`n" | Out-File $logfile -Append
    Write-Host "Check cluster" -BackgroundColor Red -ForegroundColor White
    Write-Host ""
    $summary += " : Check cluster`r`n"

    } else {

    "OK`r`n" | Out-File $logfile -Append
    Write-Host "OK" -BackgroundColor Green -ForegroundColor Black
    Write-Host ""
    $summary += " : OK`r`n"

    }

    "----------------------------------------------`r`n" | Out-File $logfile -Append

    } else {
    "`r`nERROR: Cannot connect to cluster, Please check manually`r`n" | Out-File $logfile -Append
    "----------------------------------------------`r`n" | Out-File $logfile -Append
    Write-Host "Cannot connect to cluster, Please check manually" -BackgroundColor Red -ForegroundColor White
    Write-Host ""
    $summary += " : Cannot connect to cluster, Please check manually`r`n"
    Continue

    }

    }

    $summary | Out-File $logfile -Append

    function sendemail{

    if($emailtoteam -eq "Yes") {

    $smtpserver = "smtp server hostname"
    $dateformat = Get-Date -Format M
    $emailsubject = "Daily cluster check - $dateformat"
    $to = "report recipient mailbox"

    Send-MailMessage -To $to -From "sender address" -SmtpServer $smtpserver -Subject $emailsubject -Body $summary -Attachments $logfile -Cc "copied recipient mailbox if any"

    }

    }

    sendemail

    Tuesday, September 1, 2020

    Install Splunk Universal Forwarder on Linux using Ansible

    Before automating the installation, there are some things that needs to be taken care of. 

    Universal Forwarder requires you to create a splunk administrator username and password during the installation. This can either be entered at prompt during the installation or specified in the installation command line. To get around this, we will be using a user-seed.conf that contains a preconfigured username and password as follows that can be called during the installation. More on that here https://docs.splunk.com/Documentation/Splunk/8.0.5/Security/Secureyouradminaccount

    [user_info]
    USERNAME = admin
    PASSWORD = <your password>


    Playbook is as follows.

    - name: Install Universal Forwarder(UF) agent
    hosts: ufservers
    gather_facts: no
    become: yes
    tasks:
    - name: Get previous versions of UF if installed
    shell: rpm -qa | grep splunk
    register: oldUFnamerpm
    ignore_errors: yes
    - debug:
    var: oldUFnamerpm
    - name: Remove previous version of UF
    block:
    - name: Remove old splunk from boot script
    shell: ./splunk disable boot-start
    args:
    chdir: /opt/splunkforwarder/bin
    - name: Stop old splunk
    shell: ./splunk stop
    args:
    chdir: /opt/splunkforwarder/bin
    - name: Uninstall previous version of UF
    shell: rpm -e {{ oldUFnamerpm.stdout }}
    when: oldUFnamerpm.stdout != ""
    ignore_errors: yes
    - name: Get old splunk process
    shell: netstat -tulpn | grep -i splunkd | awk '{print $7}' | awk -F/ '{print $1}' | head -1
    register: oldUFproc
    - name: Kill old splunk process
    shell: kill -9 {{ oldUFproc.stdout }}
    when: oldUFproc.stdout != ""
    - name: Remove old splunk dir
    file:
    path: /opt/splunkforwarder
    state: absent
    - name: Copy tgz from control server to remote node
    copy:
    src: /home/yinidu/Linux_UF/splunkforwarder-8.0.0-1357bef0a7f6-Linux-x86_64.tgz
    dest: /home/yinidu
    - name: Untar tgz
    shell: tar xvfz splunkforwarder-8.0.0-1357bef0a7f6-Linux-x86_64.tgz
    args:
    chdir: /home/yinidu
    - name: Move untar file to /opt
    shell: mv /home/yinidu/splunkforwarder /opt/
    - name: Copy user seed from control server to remote node
    copy:
    src: /home/yinidu/Linux_UF/user-seed.conf
    dest: /opt/splunkforwarder/etc/system/local/user-seed.conf
    - name: Install splunk
    shell: ./splunk start --accept-license --answer-yes --no-prompt
    args:
    chdir: /opt/splunkforwarder/bin
    - pause:
    seconds: 10
    - name: Stop splunk
    shell: ./splunk stop
    args:
    chdir: /opt/splunkforwarder/bin
    - name: Add splunk user
    user:
    name: splunk
    state: present
    - name: Enable boot-start
    shell: ./splunk enable boot-start -user splunk
    args:
    chdir: /opt/splunkforwarder/bin
    - name: Change folder permission
    shell: chown -R splunk:splunk /opt/splunkforwarder
    - name: Start splunk
    service:
    name: splunk
    state: started
    - name: Copy org_all_forwarder_outputs which contains heavy forwarder configuration
    copy:
    src: /home/yinidu/Linux_UF/org_all_forwarder_outputs
    dest: /opt/splunkforwarder/etc/apps/
    - name: Copy Splunk_TA_nix which contains input parameters
    copy:
    src: /home/yinidu/Linux_UF/Splunk_TA_nix
    dest: /opt/splunkforwarder/etc/apps/
    - name: Change folder permission
    shell: chown -R splunk:splunk /opt/splunkforwarder
    - name: Restart splunk
    service:
    name: splunk
    state: restarted
    - name: Set permission on /var/log
    shell: setfacl --recursive -m u:splunk:r-x,d:u:splunk:r-x /var/log
    - name: Check netstat to make sure UF is connected to the heavy forwarder
    shell: netstat -an | grep 9997
    register: netstatop
    - debug:
    var: netstatop.stdout