Posts Categorised: Blog
Making the important decisions and executing the plan
This article is the second in a series about mitigating the impact of the spectrum repack, as told by iHeartMedia Los Angeles Vice President of Engineering Doug Irwin. Read the first installment here.
Not long after this news about the Deer Park construction go-ahead came out, our SVP of Engineering Tom Cox let me know the basic framework of the project — what we wanted to achieve, and how much the budget was. It was up to me to meet the goals within the budget framework.
One question that needed to be addressed was just how long we had to put all of this together. I did reach out to local TV engineers, as I mentioned earlier, but none really had finalized plans. The best I could get was “We’ll be doing that over the summertime.”
So — since summer officially starts on June 21 — I decided that the end of May was a reasonable goal for the new site to be ready. Somewhat random, I will admit, but there was very little else to go on.
Early in the project we worked on deciding if we wanted to use two antennas fed from two separate 2-channel star combiners; or alternatively, a single antenna with a 4-channel combiner. Eventually we decided to go with the single antenna. The reasons are quite, simple really.
CBS TV used to maintain some satellite uplinks up at CBS lane, and they had a special room built in “the house” to accommodate the control gear. Sometime earlier all of it had been abandoned in place. In typical CBS fashion, it was all put together very nicely, and seemed nearly custom made for our needs.
It already had an ATS and generator outside, as well. The contacts in the ATS are limited to 150A though; and considering the ERP and antenna gain we would have to work with (for the single 8-bay antenna) the amount of current needed by the transmitters was going to fit comfortably within that limit.
If I had opted for lower antenna gain, I would have needed more transmitter power, which would have necessitated an upgrade to the electrical system. For this reason we decided to go with the 8-bay antenna.
THE TOWER ISN’T STRONG ENOUGH
Once we had the antenna picked (an 8-bay, half-wave spaced ERI Axiom) we went back to ATC and let them do the mechanical study. Unfortunately, the results were not good: The tower as-it stood was not strong enough to hold up our chosen antenna.
However, ATC has all the resources at hand to study the issue and to come up with the fixes needed. In addition, since we had picked ERI already, the two engineering teams worked together to come up with the pole needed to hold up this rather large antenna, and the more specifically, the modifications needed to the tower to make it work for the application. The pole, the tower modifications, and installation of both were on ATC’s dime.
The only real question was whether or not that part would get done on time.
CHOOSING THE COMBINER AND TRANSMITTERS
Dielectric, ERI, and Shively were all solicited for bids for the antenna and combiner. Naturally, I had to consider pricing, but with the upcoming re-pack work, I also had to consider how long the various companies would take to deliver the antenna and/or combiner. In the end I ended up choosing Shively for the combiner, and (as I already wrote) ERI for the antenna. Both had good combinations of pricing and delivery time.
The transmitter choice was pretty easy for me, since we already had two Nautel GV-series transmitters in place and on-air (for KOST and KYSR).
Clearly, Nautel isn’t the only brand that can do what it does; however, it’s easier from an on-going maintenance perspective to have the same transmitters across our stations —from spares to software updates to staff knowledge.
Working backwards (from the antenna down) and knowing the ERP we were looking for on each station, I chose two GV10s, a GV5, and a VS2.5 with the VS-HD. Yes—we were adding HD to all the stations as well. The footprints of these transmitters would also be easily accommodated with the space in the old CBS satellite room, along with three racks.
THE STL SYSTEM
Our stations on Mt Wilson each have three STLs to choose from (not including our VSAT system) and it was my intention to have all three of them also available at Deer Park.
Additionally, we have transmitter-site versions of our automation system — the idea being that they are ready to play-out audio in the event we have a system meltdown in Burbank— and I wanted that audio to be available at Deer Park as well. The reality is that I wanted to “copy” whatever I was doing up on Video Rd, and to “paste” it down at Deer Park.
At last year’s Radio Show, I was introduced to the newest member of GatesAir’s IP link family — the IP MPXp. By simply sampling the composite outputs of our current set of audio processors — with their custom settings are particular on-air qualities — we could literally do the “copy and paste” method I was after.
By using the MPXp, we would have access to our main and backup audio processors, each of which is driven by the output of a 4X1 AES selector, thus:
- Giving us access to all three STLs down at Deer Park
- And making sure the Deer Park transmitters sound identical to the Video Rd. transmitters
- By adding RDS to the appropriate inputs on the audio processors, we have access to it as well at Deer Park
So as you can see, there was no need to buy additional audio processors or RDS generators. The MPXp units are configured to use a 132 KHz sample rate, thus giving them an audio bandwidth of 60 KHz — more than enough to include all the stereo information and RDS.
CONNECTING THE SITES
Since Deer Park is .6 miles west of the main sites at Video Rd., we clearly needed some way of making them communicate. I opted for a licensed radio solution; however, the time to implement such a system ended up being more than we had to work with. For that reason, we opted to install a parallel 5.8 GHz radio link between the buildings as well.
Three of our four stations are in the Post Office building on Video Rd.; however, KBIG is next door in the Poole building. We had no fast communications between the buildings (though they are on the same AT&T switched Ethernet circuit — thus providing 10 mbps of connectivity between the two).
Since we had the IP MPXp to pass information from the Poole building to Deer Park, it was clear we needed to connect the Poole building to the Post Office before anything else. We ended up doing that with yet another unlicensed radio — this time at 60 GHz. It was an experiment that worked out well enough for us to keep it going. To be fair, this radio includes yet another 5.8 GHz radio, embedded, so that it can keep passing data even if the 60 GHz portion fades or fails.
Next time: we start putting it all together.
Doug Irwin explains what really happened to some radio stations affected by the spectrum repack
The author is vice president of engineering for iHeartMedia’s Los Angeles region and a regular contributor to Radio. This is the first in a series about how Doug and his colleagues are handling the repack.
Not long after the incentive auction came to a close last year it started to become obvious that we — iHeartMedia Los Angeles cluster — would be adversely affected by changes occurring in our immediate area on Video Rd (on Mt Wilson).
We have three stations in the Post Office building, making use of three towers that surround the building. These include:
- The KIIS-FM main, auxiliary and an aux used by our HD Radio transmitter
- The KRRL main and its aux (also used for HD)
- The KOST main antenna and its aux antenna
Additionally, immediately next door, to the west at the Poole building (now owned and operated by Insite), we have KBIG’s main and aux, and our auxiliary site for KYSR.
In the immediate neighborhood to our four stations on Mt Wilson are various FM stations, including Entercom’s 97.1 (Amp) and KRTH at 101.1. Cumulus’ KLOS (95.5) is also close by.
The TV stations directly affected by the incentive auction results are:
- KDOC, which is moving down the road to Deer Park, to the new DTV site on the tall tower
- KOCE, which is going off-air
- KJLA, which is going off-air
- KXLA, which is staying
- KCBS-TV, which is moving from RF channel 43 to channel 31, and thus changing antennas
- KTLA, which is going from RF channel 31 to channel 35, and likely changing antennas (full plan not known at this time, at least by me)
As time went by late last year, we asked the TV engineers for more details on their plans, but for the most part, they were not forthcoming. The reality is that not all of the plans were in place, so they couldn’t tell us what they were. We knew something was going to happen, we just didn’t know exactly what or when.
WOULD THE TYPICAL POWER REDUCTION BE ENOUGH TO SEE US THROUGH?
The group of stations that I’ve discussed has been very cooperative, among the ad-hoc group, with power reductions when the need arises. Many times one of the stations needed scheduled maintenance, which was usually preceded by an announcement one or two weeks in advance. Everyone prepared for it ahead of time. Occasionally, some sort of failure would necessitate an immediate repair, again accommodated well by the group, with everyone in the neighborhood going down in power by 50%. (Radio stations insisted that power reductions not occur prior to 10 a.m., and if at all possible, end by 3 p.m.)
Our corporate engineering asked about the possibility of power reductions, or worse, in the near future, to satisfy the needs of TV stations making substantial changes. I responded that the 50% power reduction had, up to that point, been good enough, and “likely” would continue to be. After all, everyone had an ox to be gored — so everyone was incentivized not to change the way we’ve been doing things. But clearly, I couldn’t guarantee the same process would always work.
Eventually, in the fall of last year (and especially after the infamous article about the repack that specifically mentioned KIIS-FM and KOST) the corporation decided that we needed to take a far more proactive approach and to build an entirely different site, in order to minimize the impact of the upcoming work.
I was tasked with finding the site.
THE DISCOVERY TRIP
Every year (almost without fail) the Southern California Frequency Coordinating Committee puts on the Christmas party at Mt Wilson, during July. Yes, a bit early, but it’s a real tradition.
Naturally, the weather is great (as opposed to Christmas time), and in 2017, representatives from American Tower Corp were on site to update the Mt Wilson crowd on what was coming up with respect to the repack.
At the same time, a couple of their guys were going to show us two sites, either of which they thought would work for our repack site: Mt Harvard, and Deer Park. Having not been to either, I very much looked forward to the opportunity.
This site is about two miles southeast of Mt Wilson at about 5,000 feet of elevation, so it covers the greater Los Angeles and Orange county area similarly. It’s not a popular site for FM, though. The major tenant is KUSC radio.
In my “discovery trip” email, I wrote the following: “This tower and antenna [shown at right] belong to KUSC radio, even though KUSC’s transmitter is up in the ATC building. It’s a curious arrangement. The building directly below this tower (which we believe belongs to ION media) has the space I mentioned above. As you can see, there’s an old transmission line running up the tower; some station removed a TV antenna evidently.” It’s a great looking tower, and there is physical space directly below that of KUSC; however, being that low to the ground could present a problem with NIER levels.
Another tower at Mt Harvard looked good but had little-to-no available space (left):
At that time I wrote: “This is the tower currently with the ION antenna. Below that you can see a 4-bay dielectric, the KLOS aux. Below that, a channel 6 basket; and below that, the old FLO antenna, which could be removed. Unfortunately this space isn’t that large or high AGL. The tower is triangular, and the most favorable leg is the one that the FLO antenna is on. So, for us, the best aperture is unfortunately adjacent to the ION antenna.” The tower looks great but there’s no space on it.
There’s also a very large candelabra-type tower on Mt Harvard (right):
I wrote: “This picture shows the structure for all the various TV antennas. Some of these are going, but it’s pretty clear that using any available space on this tower would subject us to repack work at this site.”
So, while Mt Harvard had some potential, the reality is that antenna space was at a premium (at the time) and, just like our situation on Video Rd., the TV stations didn’t have their plans in place.
The ATC guys were very interested in showing us what ATC had available at Deer Park, which is a site they bought out from CBS TV.
Formerly known as “123 CBS Lane,” Deer Park at that time was really only hosting its main tenant: KCBS-FM. It’s about .6 miles west of the main grouping of antennas on Mt Wilson; there’s an 800-ft tower there, along with a shorter one, which at one point was used by KCBS-TV for their aux antenna on channel 2, and was still in place.
At the time, I wrote: “The short tower [right] is ASR 1229046…It’s 245 feet to the very top. The last section of the tower is a pylon which is currently holding an old channel 2 batwing, obviously abandoned. What we’re thinking is that the batwing would come down, and we’d re-use the space for at least one of our antennas. Now, notice the 2-bay Jampro; that belongs to KCBS-FM.
“There’s a virtually empty building near the base of this tower — so space is abundant. The building doesn’t look great but evidently isn’t suffering too much for leaks. The roof actually looks new. There’s a generator available as well, but it appears to be only a 50 KVA set.
“If I had to pick between Deer Park and Mt Harvard, I would definitely pick Deer Park. Coverage considerations aside (since I can’t study them), I would say it would be easier, cheaper and faster to make use of this (relatively) small tower. I would also gauge that the ATC guys would prefer this solution as well, for what that’s worth.”
Next time: How I designed our new site.
One chief engineer describes the current downlink registration process as “akin to paying a more powerful mob for protection from other mobs”
As you know, all of us have been inundated with notices from various satellite program carriers about the latest fiasco with the FCC over use of C-band satellite frequencies as an ancillary terrestrial frequency band for cellphone and automotive data conveyance. Hence, the concern over interference with all of our C-band satellite program delivery from these outside sources.
What I don’t get is why do thousands of broadcasters and home C-band satellite TV hobbyists have to foot the bill for “registering” their earth stations for protection with the FCC? This to me is akin to paying a more powerful mob for protection from other mobs in old Chicago back in the 1920s.
Seems to me our tax payer dollars — along with licensing fees for every RF conveyance under the sun in radio and TV — should already be paying for this kind of protection from the big boy on the block.
Why is it that the FCC is totally unaware of the entire C band’s use for the broadcasting industry? Seems like there had to be a CFR agreement somewhere about international usage of this band for broadcaster use around the globe.
You can’t tell me the FCC was unaware of the tremendous use placed on this band for delivery of hundreds of radio talk shows, hundreds of TV shows, both network and syndicated, not to mention this is the band that local TV stations depend on to do local uplinks back to the studio when they are on live remote in their community or across the country. TV and radio networks that use this band abound, and it’s crowded with all kinds of programming, both in vertical and horizontal polarizations on dozens of transponders.
So why wouldn’t the FCC automatically protect this band from terrestrial interference as a matter of natural course? After all, it is their job, for God’s sake!
Why charge us individually to protect us from outside interference, when protecting the bands from interference is supposed to be the mission of the FCC? Did I miss something?
The author is chief engineer of Carroll Broadcasting and PD of WIOS(AM) 1480 in Tawas City, Mich.
O’Rielly called this an “artificial requirement” — but would it really hurt to have a broadcast engineer to balance out all those lawyers?
Every now and then, we broadcast engineers work with a lawyer or regulator. Contracts and regulations often require technical details; frequencies, powers, protocols, calculations, formulae, heights, etc.; and if we are “a good communicator” we can explain in a way that works what is necessary for the legal task at hand.
Usually, there is clumsy back and forth, a series of edits, and a white board or two involved. In the end, the technical types settle for “close enough.” On a good day, an Ivy League lawyer can parrot back enough sense to make the document reasonably useful.
The tribes represent different skill sets. Broadcast engineers, in particular, seldom hire plumbers or electricians or anyone else — unless the complexity, time or tools required demands it. When you take care of everything from towers to toilets, self-sufficiency is an important trait.
Lawyers, on the other hand, hire professionals to have every task done. A screw comes out in a garage door, and the “door people” are called. It’s both that putting the screw back in, in such a way that it won’t fail, is a challenge — and it is a liability. An expert in garage doors might know or see something useful that the untrained eye would miss… in any case, the liability has been transferred.
So my ears perked up when I read the following exchange (RW March 14 issue) between Paul McLane and Republican FCC Commissioner Michael O’Rielly:
RW: Do you think there should be a requirement that at least one commissioner be an engineer, as has been posited over the years?
O’Rielly: No. I don’t think an artificial requirement is the way to go. If that were a requirement, I might not have made the commission. Hopefully I’ve brought some benefit to my time here so far.
It’s the “artificial requirement” characterization that got my attention.
The commissioners have, of course, mostly had legal backgrounds. O’Rielly is actually an exception to the all-lawyers trend. Of those exceptions, I am not aware of any commissioner who has ever had an engineering background. You’d think that just once, by accident even, this would happen.
That said, I know lawyers and doctors who are hams and— gee whiz — I have cross-career skills, so a commissioner with said cross-skills would be just fine, too.
Over the years, and seemingly more recently, I’ve read a number of commissioner’s statements that just make me cringe. I’m betting you’ve read or heard some of these, also.
I think “How can someone with any understanding of how this works, come to such uniformed and thus idiotic conclusions and speak so loudly? Please lord, keep this from being reflected in regulation.”
There was a time, when “management skills” were all that was required to manage anything. It did not matter if you knew anything about trains to manage a railroad. (But it did.) Even if it was on the job training (remember that OJT?), managers could use some specialized knowledge.
One is severely limited if one is regulating or writing contracts in a vacuum.
So… I’m not saying there should be an “artificial requirement” for an engineer on the commission, but once in a while, there ought to be someone that could at least pass a ham radio test.
I can’t imagine trying that approach would put the free world at risk, at least not in quite the same way that some of the less-informed regulators and lawyers have in the past.
Think your generator isn’t important? Think again
As power outages and other major challenges continue to confront Puerto Rico more than seven months after Hurricane Maria made landfall, it’s hard to imagine any positive consequences that may have come from the storm.
But Ryan Bell of the Columbia Journalism Review highlighted one bright spot in a recent article: The natural disaster may have inadvertently reinvigorated AM radio.
He points to Mayagüez-based station WKJB(AM), which remained on air during and after the hurricane — in part because the station had learned the hard way when Hurricane Georges blew through the island in 1998. The staff had installed a backup power generator and a reinforced antenna.
[Even if your station isn’t in the path of hurricane season, Marty Hadfield can teach you about emergency preparedness — his station survived Hurricane Katrina.]
The information the station and its mostly volunteer staff put on the air was crucial during the month-plus period that television station were off the air and digital publications were off line.
Read the full CJR story online here. It’s a good reminder about the importance of emergency preparedness as we head into another hurricane season.
Without the proper attention, uninterruptible power supplies can create more problems than they solve
LOS ANGELES — Uninterruptible power supplies are a necessary evil at transmitter sites and rack rooms. Without the proper attention, UPSs can create more problems than they solve. Let’s take a look at some ways to minimize those potential problems.
SINGLE POINTS OF FAILURE
A single point of failure is generally one piece of equipment, the failure of which “kills” the radio station. Often, single points of failure are found in inherited systems, but sometimes you will inadvertently create one when installing new gear without a clear plan. UPSs easily fall in to this category because they often just get racked up in order to “protect” a new piece of gear that is particularly susceptible to power hits. They are added to a rack, plugged into raw AC power, and then the new gear gets plugged into the back of the UPS, and we’re on our way back to the studio or home. Later, more equipment gets plugged in, and then something else, and on and on.
Of course, the nightmare scenario is that the main and backup systems get plugged into the same UPS, and then it conks out on you — not because of the load, but because of a component failure, or more likely, battery failure. Say you plugged in two STLs or two audio processors to the UPS. It fails, and then you’re left with nothing but dead-air until you get there to move the AC power cords around.
STRATEGIZE AC POWER SOURCES
One way to get around that scenario is by using multiple UPSs and being certain that all critical gear is plugged into different units. Consider these steps:
- Have one UPS per rack. If you have multiple racks, consider having a UPS at the bottom of each rack. Also make sure that each UPS is fed from a separate AC circuit breaker. You wouldn’t want a single breaker trip or a failure to kill power to more than one rack, right?
- Don’t connect main and backup systems to the same UPS. Engineers organize stuff in racks in different ways. I like to separate main and backup systems into different racks, which is why I place one UPS per rack. It’s easier to organize power feeds in this fashion.
- Main and backups in the same rack call for clearly labelled power strips. If you have only one rack, or you have mains and backups in the same rack already, then install a specific AC outlet strip fed by a separate UPS and plug backups into that, instead.
- Keep mains on UPS. Keep backups on raw power. If you simply don’t have room for two UPSs, then keep backups plugged into raw power instead. Make sure your remote control, used for the switching, isn’t plugged into the UPS.
TWO TYPES OF UPSs
You likely know there are two types of UPSs: those that only switch over to battery power when there is a power failure, and those that are online all of the time. I prefer the latter because I find that the former allow surges and brownouts to pass right through (depending on the length of the event, of course).
Online UPSs use double-conversion, meaning they take the AC power, rectify it, charge the batteries, and then use the DC to generate a sine-wave output. There’s no switchover time because the batteries are effectively the DC source for the 60 Hz oscillator.
One advantage to the off-line type is that the batteries can be easier to swap out. Keep that in mind. An example of the online type would be those made by Falcon electric, and a typical off-line brand would be APC.
Remember to label new UPSs with information about when they were new or when new batteries were installed. It’s a good idea to note this information in your maintenance records, too. My rule of thumb is replace batteries in off-line UPSs after three years.
Every radio station should have, at minimum, two ways to get program audio to the transmitter site
LOS ANGELES — An STL system is just as crucial in a radio station’s air-chain as the transmitter or antenna. After all, if there’s no audio to transmit, then even the best transmitters and antennas are of little use. No one tunes in to hear dead air.
Every radio station should have, at minimum, two ways to get program audio to the transmitter site. A radio STL at 950 MHz is a good option for one of the two means. “Wireline” of some sort is a good choice as well. (By wireline, I mean any type of connectivity provided by the local exchange carrier—whether audio circuits, T1s, or metro Ethernet.) Technology diversity is the best way build redundancy in to your STL system. Your primary and auxiliary STLs should not both be of the same type; if you use “wireline” for the main, use radio for the aux, and vice-versa.
Let’s take a look at the reasons for this assertion of mine.
PROBLEMS WITH RADIO LINKS
While private radio links make great STLs they are subject to problems, including these:
- Outright equipment failure
- Antenna/transmission line failure
- Interference that just shows up one day
- Deep fades
Let’s talk about the mitigation of these issues one at a time.
Equipment failure. This is the most likely means by which you will “lose” your STL. Clearly the best way to get around this is to have a standby transmitter, and a standby receiver, both of which can be switched ON using remote access. The receiver part is easy—derive two separate antenna feeds, drive the input of both receivers, and then put a switch on the outputs so that either can feed your air-chain. The transmit side is a bit more difficult—since they both can’t be radiating at the same time. The most practical approach is to use an RF switch to select the output of one of the two transmitters to drive the transmit antenna. Another (better) approach is to have separate transmit antennas. No RF switch is necessary then, but you’ll still need a way to turn one transmitter ON and have the other OFF, and the ability to reverse their roles.
[Related — “Best Practices: Value Engineering “]
Antenna and/or transmission line failures. Another way radio STLs fail is by way of bad connections on transmission lines and antennas. Often they sit out in the weather for years, and if they were not installed correctly to begin with, problems can show up several years down the road.
If you have two receivers, ideally they can be fed by separate antennas; that way, a failure in the receive antenna system won’t leave you with two non-working receivers. Separate feeds are best, even if one is from an inferior antenna.
Likewise, on the transmit side, having a choice of antennas provides an advantage. Even if you have two transmitters, having a bad transmit antenna will leave you with little prospects. This type of problem often shows up at the worst time as well—say for example in the winter—so getting someone out to fix it might not be as easy.
New interference. 950 MHz-band resources are typically shared amongst multiple users in a market. The unfortunate reality is that a radio system that is working fine one day can sometimes be interfered with the next, leaving you scratching your head. New systems come up (though you should know about any of them through the Prior Coordination Notification protocols) and create problems unexpectedly; a random transmitter in the field might develop spurious emissions that accidentally show up on your channel; and there’s always the possibility of newly developed intermodulation products at the receive site. If you are using a digital system, there’s also the possibility of desensitization of your receiver, because of strong, local carriers that are nowhere near your own frequency.
These types of problems are rarely solved in a day and the best way to be prepared is by having one or two other means by which program gets to the transmitter site.
[Related — “Best Practices: Krone Blocks “]
Deep fades. If a path is poorly designed, or if it has other problems, none of which had come to your attention previously, you might experience signal fades on the path that are deep enough to make your receiver “mute” causing dead air. When you first start at a station you should take note of RSSI levels on the receiver(s) and also forward and reflected power readings on the transmitter(s). Sometimes transmission line problems can make the signal weak at the far end (though still receivable) while not knocking it all the way out (i.e., fade margin is unexpectedly used up). Antennas that are not aligned properly can cause similar problems.
Radio plus wireline strategy. So, to reiterate my earlier assertion, technology diversity is the best way build redundancy in to your STL system. There’s simple logic behind this idea.
While there are four primary ways to have a radio system knocked out (see above) none of them is related to the performance of “wireline.” If you experience any of the four you can switch over the wireline and keep going while you figure out your radio problems.
Conversely, wireline problems, when they happen, will be totally unrelated to any 950 MHz system performance. Wireline issues would include fiber cuts, power outages, and other miscellaneous central office equipment failures.
One exception I will point out is the use of part 101 radio systems. It would be acceptable to use 950 MHz and 11 GHz radio systems as main and auxiliaries.