Why do some leaders struggle with empathetic leadership?

The world isn't black or white when it comes to anything, including leadership styles.  I have worked with many leaders who try to be empathetic but for a number of reasons can't realize these behaviors in practice. Some of the reasons this is so difficult is because when many of us were younger, we may have brought part of themselves to work.  Today, people are encouraged to bring their whole authentic self to work.  Leading people, managing resources, following laws, rules, and achieving objectives and outcomes is challenging within itself as a baseline. Learning about George struggling through his transition to Jenny or Matt's realization that his mother hates him or Ann is pregnant or Jeff's wife is intermittently fasting and he is suffering because he can't eat or the myriad of issues that have nothing to do with getting work done is hard work.  It is absolutely important to invest time in people and to be empathetic as a leader but it takes a lot of time and energy.  There is no "right" way to do it.  Some are good at it and some can't handle it because it is just too much for them.  Highly technical leaders with a propensity for data or numbers may struggle with the soft side.  Today, there is a lot of soft side ..   

I think that’s the old school mentality that strength comes from not showing a sign of vulnerability. The opposite is true.

As Esther mentioned below, not all who end up in a leadership role should be leaders. There are objective ways to assess this, but most won't go through the effort - instead we promote the top seller to sales manager and assume they will "figure it out" to be a great leader. When it comes to empathy specifically, most of us were not taught what empathy is or how critical it is in leadership. 

Within technology organizations, often the top individual contributors are those that are promoted into leadership. They haven't been able to develop leadership skills and focus heavily on task management and just getting the work done. I feel as though they are measuring those they lead against their memory of how they have done it in the past. The patience necessary to allow others to grow and struggle doesn't exist, with business constraints sometimes contributing to that lack of patience. 

Conversely, when prospective leaders are given learning experiences within leadership and are given the training to develop that empathetic leadership, they learn better how to balance the need to hit targets against the need to develop a team. They learn to have a longer-horizon viewpoint.

I think this is a generational thing

Millennials and Gen Z generally have more empathy as they realize that a job is a job, and that you are more than a number on a page. Older generations had much more of a numerical focus on delivering value, and it used to be the case that 'hard work pays off'

However, in today's climate, hard work doesn't pay off for the majority of workers. Hard work and going 'above and beyond' just lead to more work being added without the additional benefit.

Because of this, empathetic leadership is on the rise, as more people are in the same mindset and bucket. People want to be cared for, so they care for other people.

Though unfortunately, most folks in a senior leadership positions see their workers as ants, more than other humans with rich complex lives