Netapp Filer error "retrying all the portals again, sincethe portal list got exhausted"

One of the ESX host which is hooked up to software iSCI was throwing following iSCSI related messages into VMKernal :

Mar 11 05:10:17 esxhost vmkernel: 4:12:19:31.755 cpu0:1076)iSCSI: session 0x35203f90 connect timed out at 38997177 Mar 11 05:10:17 esxhost vmkernel: 4:12:19:31.755 cpu0:1076)<5>iSCSI: session 0x35203f90 iSCSI: session 0x35203f90 retrying all the portals again, sincethe portal list got exhausted Mar 11 05:10:17 esxhost vmkernel: 4:12:19:31.755 pu0:1076)iSCSI: session 0x35203f90 to iqn.1992-08.com.netapp:sn.118062462 waiting 60 seconds before next login attempt Mar 11 05:11:17 esxhost vmkernel: 4:12:20:31.756 cpu0:1076)iSCSI: bus 0 target 0 trying to establish session 0x35203f90 to portal 0, address 10.100.200.137 port 3260 group 2000 Mar 11 05:11:32 esxhost vmkernel: 4:12:20:46.756 cpu1:1074)iSCSI: login phase for session 0x35203f90 (rx 1076, tx 1075) timed out at 39004677, timeout was set for 9004677 Mar 11 05:11:32 esxhost vmkernel: 4:12:20:46.756 cpu0:1076)iSCSI: session 0x35203f90 connect timed out at 39004677 Mar 11 05:11:32 esxhost vmkernel: 4:12:20:46.756 cpu0:1076)<5>iSCSI: session 0x35203f90 iSCSI: session 0x35203f90 retrying all the portals again, sincethe portal list got exhausted

As per NetApp, this is somewhat expected behavior from the array itself; as the filer is designed to provide default access through iSCSI to any hosts configured to access any portal groups;ESX
host's iSCSI initiator logs into the filer's target IP address.Using Dynamic discovery, the ESX host issues a “SendTargets” command to the filer. Filer responds with “ALL available” target portals which by default includes any network interface with iSCSI enabled (this is the default behavior). ESX host sees this list of portals and attempts to establish connection on them . Does it randomly select from the available target portals, or does it have some logic built in about how to determine which portal to try first? ESX host attempts to log into a target portal to which it has no network route. This causes the errors to be generated in the “vmkernel” logs. So, in order to fix this problem from the filer perspective, we added an accesslist for the iSCSI initiator node name of the ESX host. Basically, by default an iSCSI initiator has access to all available portals, as I mentioned above. In order to limit the target portals that are reported back to the ESX host,
we create an access list entry that states which iSCSI target portals are available to a given iSCSI initiator node name.

Update/Remove Network card for vSwitch

There may the time when you have to update/remove vSwitch NIC with some other NIC
To do that run following command which will add vmnic0 into vSwitch0
esxcfg-vswitch -U vmnic0 vSwitch0

This will remove vmnic0 from vSwitch0
esxcfg-vswitch -L vmnic0 vSwitch0

P2Ving IIS windows server in DMZ

Please read this before you attempt .

We had been attempting to virtualize one of the dying old hardware hosting business critical application in DMZ. This was my first experience virtualizing web server in DMZ. So I learn too many things to from this P2V effort.

1. When you virtulize any web server in DMZ please involve Firewall/Network Admin.
   
2. Ports need to be open between ESX host which will be hosting that physical machine + Physical machine + VC client box which is performing this effort.
   

ESX Host IP -> (All ports) Physical server IP -> (All ports)VC client server IP .

As you can see we have open all the ports and reason behind doing this was to avoid any port related error while doing P2V. This is safest approach from P2V and security prospective.

3. Before virtualizing take backup of IIS application using IIS console. Once backup completed shut down all the IIS service and any other services like Anitvirus. Also run ipconfig /all > ip.txt to note down all the IP addresses.
   

4. Use convertor and virtualize the physical box and choose not to start and install vm tools.
   

5. Once this box is virtualize remove all the unwanted hardware using following link.
   
6. Make sure you have removed the vm NIC card and then power on the system. The reason we do this to have a neat and clean system before adding NIC. This also safeguards from any IP or application conflict.
   
7. Once the machine is powered ON check if any NIC teaming was done. I had a great difficulties because I used this link and deleted the hidden driver before removing it from NIC.
   
8. Remove the NIC team because IP address are assign to the team in case of Teamed server not to the individual NIC adapter.
   
9. Once the team is removed from the NIC then uninstall the NIC team software and rest all software (Hardware related) which is not required. Please follow the reboot sequence as machine request you. I thought I will reboot it once for all and landed up in trouble. I think we should have sometime listening attitude towards Mr. Gates creativity.
   
10. Now follow the link to remove hidden NIC adapter imported as a part of P2V.
    
11. Once all unwanted adapter is removed then proceed with installing VM tools.
    

12. After this add VM nic and then power on machine and assign IP address.

There may be the case where you have to restore the IIS or if your application is DOT Net based then you have to reinstall. In my case I have attempted thrice to P2V same IIS server thinking that either application or P2V got corrupted. But finally it was .Net which was causing the whole problem. Reinstalling fixed the problem with the application. Not sure why .Net got corrupted

P2V Enterprise Convertor V1.0

When P2V happen it create a disk based on how many physical disks are present in our physical machine. I was under impression that Enterprise convertor does improved from legacy vmware converter but I am wrong. Hopefully next release will have this feature.

Snapshot and disk expansion

I learned my snapshoot lesson in very hard way. So I would recommend when there
are many VMware admin with little knowledge please be careful with expansion or stuff like that.Today one of the VM’s had snapshoot and I did not realize it (Yes I am stupid Idiot). It has only C: drive which needs to be expanded. So I ask one my colleague to use GUI and perform the expansion. He tried and the value got reverted. What I meant was when I changed size from 5GB to 6GB it will not accept that changes

But if the machine does not have snapshot then it will accept these changes. So this is a indication that it has It is like a new discovery for me