Restarting hostd service on the host

Sometime restarting management service on the host will not restart the “hostd ” service. It is in hung state or the script gets stuck when stopping the service. It may happen that the SSH session to the ESX host becomes unresponsive. To restart this without restarting the host follow below steps. This has been documented on KB for VMWare

1. Navigate to the /var/run/vmware directory:
# cd /var/run/vmware
2. List the files vmware-hostd.PID and watchdog-hostd.PID:
# ls -l vmware-hostd.PID watchdog-hostd.PID
3. Determine the Process ID (PID) management service, view the contents of the file vmware-hostd.PID:# cat vmware-hostd.PID
4. For example, [root@vmware]
# cat vmware-hostd.PID
1191[root@vmware]#
5. Use the resulting PID to kill the process. Caution: Type the kill -9 command carefully, it kills the process of the supplied PID without exception or confirmation.
# kill -9 1191
6. Delete the vmware-hostd.PID and watchdog-hostd.PID files:
# rm vmware-hostd.PID watchdog-hostd.PID
7. Restart the management service, run:
# service mgmt-vmware restart

How To Decide Static Mac address for Virtual machine

This is useful for software that is licensed by the MAC address of the server it is running on or if you migrate a physical server to virtual and want to keep the same MAC address. A VM’s MAC address will change if the location of the VM changes, ie. different path on the same host. During a hot migration (Vmotion) the location of the VM does not change, this is also true of a VM that changes hosts due to HA/DRS migration. During a cold migration the location of the VM does change so the MAC address of the VM will change.
MAC addresses are hex values and consist of six groups of hex numbers. The first 3 octets of the MAC address is a unique code assigned to each NIC vendor, this is also called the Organizationally Unique Identifier (OUI), VMware’s OUI is “00:50:56”. The last 3 octets are assigned to all of the NIC’s for that vendor. Each MAC address must be unique to avoid conflicts with other network devices. You do not have to use the VMware assigned range (00:50:56) for your NIC’s. If you are coming from a physical server you can use the previous NIC’s range instead, (ie. HP/Compaq NIC’s use 00:08:02) as the first 3 octets. Just make sure the physical NIC and virtual NIC with the same MAC address are not active on the same physical network at the same time. There are several methods for setting a static MAC address on a VM which are listed below. If you use the first method you must stay inside VMware’s allowed MAC addresses (00:50:56:00:00:00 – 00:50:56:3F:FF:FF) or the VM will not start. You need to maintain the pool of this MAC address to avoid conflict

Trying to be Photographer Part8

Aperture, ISO and Shutter Speed - The Good Kind of Threesome

PS: I have to admit that I have stolen this from blog site fearing that this information might go sometime

 . I would like to thanks Choln for wonderful tutorial.

Canon 24-70mmL, f/2.8, ISO 1600, 0.125s

I thought I would take a little time and whip up an article about some photography basics. Please, please note the use of the word “basics” in that last sentence. If you’re been shooting for any appreciable length of time then you’re probably not going to get too much out of this. But given that we’re smack in the middle of the present giving time of year, it’s likely that in a few days there’s going to be a glut of new DSLR owners out there. What I’m aiming for is the article I wish I’d found when I first got my camera, to help explain the relationship between three critically important parameters: the ISO speed, aperture (or f-stop number), and shutter speed.

To lead with, the easiest way I think to approach these three things is to understand physically what they mean inside your camera. So, here we go.

• ISO speed: A measurement of how quickly your camera’s sensor absorbs light.

• Shutter speed: A measurement of how long your camera’s shutter is open for when you take a shot.

• Aperture: A measurement of how wide the shutter on your camera’s lens opens up when you take a shot.

For completeness I should note that ISO speed is not specific to digital cameras with sensors. On film cameras, a particular roll of film will have a specific ISO speed that it is rated for. Fortunately for DSLR users like us, we can change the ISO speed without having to change film. All three of these parameters affect how light gets to your camera’s sensor to create an image, so let’s start looking at them in more detail.

ISO Speed

The ISO speed changes how quickly the sensor absorbs light. Lower numbers mean less quickly and higher numbers mean more quickly. Now, the thing to keep in mind is that at high ISO speeds, you will introduce some graininess into your image. Therefore, the basic rule of thumb is that you want to shoot at the lowest ISO speed that you can get away with given the environment you are in. For example, if you are outdoors in the middle of a sunny day, you can probably shoot at a very low ISO speed like 100 or 200. Conversely, if you are shooting indoors or at night (or both) you will probably need to shoot at a high ISO speed in order to properly expose your image.

Shutter Speed

This one is probably the easiest of the three to get your head around. It’s exactly what it sounds like: the length of time that the shutter is open and light can get in through the lens to your sensor. Shutter speeds on Canon cameras are displayed as inverse numbers against one second. So, if your camera’s shutter speed is set to 125, that means that the shutter will be open for 1/125th of a second. Therefore, higher numbers mean that the shutter is open for less time. The effect this has is on the sharpness of the image. If the shutter is open for a very short amount of time, then even if the thing you are shooting is moving you will be able to get a sharp image of it, because it won’t have time to move around before the shutter closes again. The downside is that if the shutter is only open for a instant, then not a lot of light can get in to the sensor.

Aperture (or F-Number)

The third and final member of our roster here is the aperture. This is a measurement of how wide the shutter opens up when you take the shot. In terms of the F-Number, it can be a little bit confusing at first. You’ll see lenses listed with specs that read like “35mm f/2″ which doesn’t mean too much at first, so let’s take a look at that. The “35mm” part is the focal length of the lens, in this case fixed at 35 millimeters. Now, the “f/2″ part means that the ratio of the focal length to the maximum width that the shutter can open up to is 2. Confused? Let’s use some real numbers. This is saying that on this 35mm focal length lens, the maximum diameter that the shutter can open up to is 17.5mm. So, the smaller the F-number is, the wider the lens can open up relative to its focal length.

Understanding the effect this has on your image is a little more complex than with the others because two different effects come into play. The first is the depth of field in your shot. I’ve personally always thought that the term “depth of field” is a bit, well, wrong for what it’s used to describe. In my opinion, a more accurate description would be “plane of focus” so I’m going to run with that. Here we go. Imagine you are taking a picture, and that there is a magical plane of glass somewhere in front of you that is parallel to the front of your lens. “Magical” because it can pass through other objects, like the ones you’re taking pictures of. Everything within the plane of glass will be in focus, and things that are in front of or behind the plane of glass will be blurred. Got that? Good. Now here’s the neat trick: the wider your lens opens up when you take the shot (low F-number), the thinner that plane of glass becomes. Conversely, if your lens opens up only a teeny amount (high F-number), then the plane of glass becomes very very thick. So, a very high F-number means that probably everything in your shot will be in focus. A very low F-number means that only a thin plane will be in focus in your shot.

Still not quite making sense? It’s a little hard to visualize at first. Fortunately, Ryan’s taken a great shot at a wide aperture that should help make things clearer.

Canon 100mm macro, f/4, ISO 400, 0.001s

As you can see, with a wide aperture, the caterpillar is in focus but the areas both in front of and behind it are a bit blurred out. This is often used to make objects in the foreground pop into sharp relief by blurring out the background, which is a standard portrait shooting technique shown here.

Canon 50mmL, f/2, ISO 1000, 0.013s

So the first of the two effects is related to which parts of your shot are in focus and which aren’t. The second again relates to light. The wider your lens opens up, the more light gets in to the sensor. So shooting with a wide open lens can help you in low-light situations. On the other hand, if you have tons of light, then you can shoot with much smaller apertures if you so choose.

Putting it Together

Now you should have a good understanding of what these three numbers are actually doing on your camera. Let’s talk a little bit about how the three are interrelated. All three of them share a common trait, namely that they affect how light gets into your camera to produce an image. This is one of the reasons that you’ll read over and over that lighting is the most important thing in photography. Let’s look at a few scenarios, starting with a situation in which you are doing social shooting in very low light. In this case, you will typically want to set your ISO speed very high at something like 1600, and you’ll want to open your lens up as wide as it will go (low F-number.) Then, you will set the shutter speed as fast as you can get away with while still getting a reasonable exposure. A good rule of thumb for the shutter speed is it’s the inverse of the focal length of your lens. Meaning, if you’re shooting with a 50mm lens, try to have the shutter speed 1/50th of a second or faster. Practically, I get away with a slightly slower shutter speed than this rule dictates all the time, but it’s a good thing to keep in mind.

Canon 50mmL, f/1.2, ISO 1600, 0.025s

You definitely see the grain here due to the high ISO setting of 1600. Also, you’ll note that while the subject’s left eye is right in focus, her hair over her right shoulder is a bit blurred. This is due to the very wide aperture of f/1.2. The shutter speed was 1/40th of a second, and fortunately she wasn’t moving very much so I was able to get a sharp shot. This was despite the fact that there were only a few tea light candles to illuminate this frame.

Let’s look at a more extreme kind of shot next. Low light, but where you want a low ISO to avoid the grain, and a reasonably small aperture so things are in focus. If that’s the situation you are in, then in order to get enough light into the camera to get the image, you have to have a very, very slow shutter speed.

Canon 17-85mm, f/8, ISO 100, 25s

In the shot above, taken by Ryan, the ISO speed is very low at 100, and the aperture is at f/8. In order to get the exposure correct, the shutter was open for 25 seconds. Obviously, in order to do this effectively, you need to be using a tripod or some sort of stabilizing device to keep the camera still so that the image comes out sharp.

Let’s look at one more that’s not in such an extreme low-light situation.

Canon 24-70mmL, f/2.8, ISO 100, 0.001s

This photo was taken in the late afternoon when there was a reasonable amount of ambient light out. The ISO speed is set very low at 100 so the image isn’t grainy. I have the aperture open quite wide at f/2.8, so Demian’s head is in focus but the background is nicely blurred out. The shutter speed was 1/1000th of a second in this shot, so I didn’t have to worry about movement from Demian or the camera affecting the sharpness of the shot.

Do I really have to worry about all of this?

At this point hopefully you have a better understanding of why these three magic numbers are important for your photography. Nonetheless, you may also be there thinking “Seriously? Do I have to worry about all of this stuff all the time?” The answer in this day and age is “no, you can make the camera do at least some of the work for you.”

If you’re just getting into your DSLR, you’ve probably noticed that it has a variety of different shooting modes. When I first started learning about all of this, my more experienced friends advised me to use Aperture Priority mode when taking pictures. I recommend that you do the same. In Aperture Priority mode on my Canon rig, you select an ISO speed and the aperture to use, and the camera will pick out the shutter speed for you. This is convenient because you don’t have to constantly readjust the shutter speed as you’re trying to get your shots. There’s also a Shutter Priority mode, which I’ve barely ever used, and it does more or less the opposite. In that mode, you pick the ISO speed and the shutter speed, and the camera will select the aperture for you. I know that on some cameras like the new Nikon D300 there are modes that will also select ISO speeds for you, but I don’t have a rig that has this functionality personally so I don’t know too much about how to use it.

At any rate, shooting in Aperture Priority mode is a great way to start out since it lowers the number of things you need to concern yourself with. And you can always see what the camera is recommending for you in the viewfinder, or by looking at the EXIF data later on when you’ve processed the image. I’ve personally been shooting in fully manual mode a great deal as of late, but that’s mostly because I keep finding myself in very dark situations and I’m generally insistent on getting a sufficiently fast shutter speed so that the shots are clear. That said, for normal daytime shooting, myself and many of my friends use Aperture Priority mode a great deal of the time.

In Conclusion

I hope this serves to clarify some of the jargon that you may be absorbing. There’s definitely a learning curve with photography, but once you get the basics down it’s an incredibly fun endeavor. If anybody has questions please ask them via comments and I’ll do my best to answer them. Happy Holidays!

Best practices for deploying Citrix on vmware ESX

There is a discussion going on dutch site about how to run citrix on VMWare. They came up with conclusion that if we disable balloon driver on the vm's which get installed as a part of the vmtools it will perform better. KB article from VMWare explains how to disable the balloon driver


The goods:
Virtual Infrastructure 3
Windows 2003 Std (or Enterprise) Edition R2 (x86, not x64)
Citrix Presentation Server 4.0 (yes, I know, the old one ;))
The tips:
First this: it all depends on the applications used! Context switches is the key here...
Use Windows 2003, not Windows 2000
Don’t P2V your servers, but use clean templates
Make sure the correct HAL (single or multi) is installed in the virtual machine. Otherwise, your vCPU will spike.
Always assign 1vCPU. If necessary, add a 2nd vCPU. Do not use 4 vCPUs!
Use 2 GB to start. Scale up to +-4 GB of vRAM if necessary
Use 1 .vmdk for your system partition (C:\ or other remapped drive letter) and 1 separate .vmdk for your program files.
Put the page file on the 2nd .vmdk
Important: disconnect any .iso file in your virtual CD-Rom
Use roaming profiles and cleanup your profiles at logoff
Disable sound for your published apps
Install the UPH service (download it here)
User sessions: for me, 30 users on a VM is the sweet spot. Do not expect to get as many users on it as on a physical box!
Scale out, not up. A major advantage of VM is to clone/NewSID/sysprep existing servers and put them into your existing Citrix farm. Just stop & disable your IMA service, clean up your RMLocalDB (if you use enterprise) and NewSid the thing. Refer to this support article for more info.
Use dual core or quad core systems. This because ESX will have more CPU to schedule its vCPUs on.
Don’t ever use a 2 vCPU Citrix virtual machine in a 2 pCPU physical machine!
Do not install the memory ballooning driver while installing the VMware Tools
Do not use a complete installation Vmware tools: there is an issue with roaming profiles and the shared folders component. See my previous article for more info.
Disable COM ports, hyperthreading, visual effects & use speedscreen technology where possible.
Use snapshots when installing applications or patching your servers (yes! With VMware you can do this!). In case of disaster, you can still revert to the original working server without using backups. Make sure all snapshots are removed ASAP when finished!
Always check that there are no snapshot leftovers (f.e. the infamous _VCB-BACKUP_ when using VCB)
Don’t forget you can use DRS rules to run your Citrix servers on separate physical hosts.
Check out this vmworld 2006 presentation
And last but not least: do not forget to read ESX's (excellent) performance tuning white paper.

Scalable Storage Performance with VMware ESX Server 3.5 - VMware VROOM!

VMWare came with the performance blog about “Scalable Storage Performance with VMware ESX Server 3.5” But I have some serious question about the way they concluded this. I asked them

Your statement “The maximum supported value is most commonly 256. For an I/O group (ESX Server(s) – LUN), it is important that the number of active SCSI commands does not exceed this value” what does it mean in terms of VMDK? How many VMDK I should be placing so that it should not exceed this value? My understanding about the queue depth is I can change the queue depth at the host level to match that with Frame and I can still push that many I/O from the host. For example if we see the queue depth of Frame around 1000 and if you have set the host at around 540 then you can get into potential problem. I can see some SCSI abort into my vmkernal logs. To fix this I can change the queue depth at host and that can take care of this error.

I also would like to know what was the block size of the VMDK and was it aligned during your test ?

We at VMware often get questions about how aggressively physical systems can be consolidated. Scalability on heavily-consolidated systems is not just a nice feature of VMware ESX Server, but is a requirement to support demanding applications in modern datacenters. With the launch of VI3 with ESX Server 3.5 we’ve further improved the efficiency of our storage system. For non-clustered environments, we’ve already shown in this comparison paper that our system overheads are negligible compared to physical devices. In this article we’d like to cover the scalable performance of VMFS, our clustered file system.

ESX Server enables multiple hosts to reliably share the same physical storage through its highly optimized storage stack and the VMFS file system. There are many benefits to a shared storage infrastructure, such as consolidation and live migration, but people commonly wonder about performance. While it is always desirable to squeeze the most performance out of the storage system, care should be taken not to severely over-commit the available resources, which can lead to performance degradation. Specifically, the primary factors that affect the shared storage performance of an ESX Server cluster are as follows:

1.The number of outstanding SCSI commands going to a shared LUN

SCSI allows multiple commands to be active on a link, and SCSI drivers support a configurable parameter called “queue depth” to control this. The maximum supported value is most commonly 256. For an I/O group (ESX Server(s) – LUN), it is important that the number of active SCSI commands does not exceed this value, otherwise the commands will get queued. Excessive queuing leads to increased latencies and potentially a drop in throughput. The number of commands queued per ESX Server host can be derived using the esxtop command.

2.SCSI reservations

VMFS is a clustered file system and uses SCSI reservations to implement on-disk locks. Administrative operations, such as creating/deleting a virtual disk, extending a VMFS volume, or creating/deleting snapshots, result in metadata updates to the file system using locks, and hence result in SCSI reservations. A reservation causes the LUN to be available exclusively to a single ESX Server host for a brief period of time. It is therefore preferable that administrators perform the above-mentioned operations during off-peak hours, especially if there will be many of them.

3.Storage device capabilities

The capabilities of the storage array play a role in how well performance scales with multiple ESX Servers. The capabilities include the maximum LUN queue depth, the cache size, the number of sequential streams, and other vendor-specific enhancements. Our results have shown that most modern Fibre Channel storage arrays have enough capacity to provide good performance in an ESX Server cluster.

We’re glad to share with you some results from our storage scalability experiments. Our hardware setup includes 64 blades running VMware ESX Server 3.5. They are connected to a storage array via 2Gbps Fibre Channel links. All hosts share a single VMFS volume, and virtual machines running IOmeter generate a heavy I/O load to that one volume. The queue depth for the Fibre Channel HBA is set to 32 on each ESX Server host, which is exactly how many commands are configured to be generated by all virtual machines on a single host. We measure two things:

•Aggregate Throughput - the sum of the throughput across all virtual machines on all hosts

•Average Latency - the end-to-end average delay per command as seen by any virtual machine in the cluster

It is clear from Figure 1 that except for sequential read there is no drop in aggregate throughput as we scale the number of hosts. The reason sequential read drops is that the sequential streams coming in from different ESX Server hosts are no longer sequential when intermixed at the storage array, and thus become random. Writes generally do better than reads because they are absorbed by the write cache and flushed to disks in the background

Figure 2 illustrates the effect of commands from all ESX Server hosts reaching the shared LUN on the storage array. Each ESX Server host generates 32 commands, hence at eight hosts we have reached the recommended maximum per LUN of 256. Beyond this point, latencies climb upwards of 100 msec, and could affect applications that are sensitive to latencies, although there is no drop in aggregate throughput.

These experiments represent a specific configuration with an aggressive I/O rate. Virtual machines deployed in typical customer environments may not have as high a rate and therefore may be able to scale further. In general, because of varying block sizes, access patterns, and number of outstanding commands, the results you see in your VMware environment will depend on the types of applications running. The results will also depend on the capabilities of your storage and whether it is tuned for the block sizes in your application. Also, processing very small commands adds some compute overhead in any system, be it virtualized or otherwise. Overall, the ESX Server storage stack is well tuned to run a majority of applications. If you are using iSCSI or NFS, this comparison paper nicely outlines how ESX Server can efficiently make use of the full Ethernet link speed for most block sizes.We’re always pleased to show the scalability of VMware Infrastructure 3, and the file system that supports the VI3 features is a good example. Look for more details on storage and VMFS performance in the form of whitepapers and presentations from VMware and its partners in the coming weeks.