Until recently, software-defined networking (SDN) technologies have been limited to use in data centers — not manufacturing floors.

But as part of Intel’s expansive plans to upgrade and build a new generation of chip factories in line with its Integrated Device Manufacturing (IDM) 2.0 blueprint, unveiled in 2021, the Santa Clara, Calif.-based semiconductor giant opted to implement SDN within its chip-making facilities for the scalability, availability, and security benefits it delivers.

“Our concept was to use data center technologies and bring them to the manufacturing floor,” says Rob Colby, project lead. “We’ve had to swap the [networking infrastructure] that exists, which is classic Ethernet, and put in SDN. I’ve upgraded a whole factory from one code version to another code version without downtime for factory tools.”

Aside from zero downtime, moving to Cisco’s Application Centric Infrastructure (ACI) enabled Intel to solve the increasingly complex security challenges associated with new forms of connectivity, ongoing threats, and software vulnerabilities. The two companies met for more than a year to plan and implement for Intel’s manufacturing process security and automation technology that had been used only in data centers.

“This is revolutionary for us in the manufacturing space,” Colby says, noting the cost savings from not taking the factory offline and uninterrupted production is a major financial benefit that keeps on giving. 

That ability to upgrade the networking infrastructure without downtime applies to downloading security patches and integrating tools into the production environment alike, Colby adds.  

“Picture a tool being the size of a house. One of our most recent tools is a $100 million tool, and landing a tool of that size involves a lot of complexity, after which I have to connect it so it can communicate with other systems within our infrastructure,” Colby says. “[Having SDN in place] makes landing tools faster and the quality increases. We’re also able to protect it at the level we need to be protecting it without missing something in the policy.”

Bringing SDN to the factory floor

The project, which earned Intel a 2023 US CIO 100 Award for IT innovation and leadership, has also enabled the chipmaker to perform network deployments faster with 85% less headcount.

Colby says it took a couple of years for the partners to build the blueprint and begin rolling out the solution to existing factories, including rigorous offline testing before beginning.

The migration required no retraining of chip designers in the clean room but some training for those in the manufacturing facilities. “We really went above and beyond to make it as seamless as possible for them,” Colby says. “We’ve recently been testing being able to migrate them over to ACI on the factory floor without any downtime. That will accelerate our migration for the rest of the factory floor.”

The collaboration with Cisco enables ACI to be deployed for factory floor process tools, embedded controllers, and new technologies such as IoT devices being introduced into the factory environment, according to Intel.

It was “clear that we needed to move to an infrastructure that better supported automation, offered more flexible and dynamic security capabilities, and could reduce the overall impact when planned or unplanned changes occur,” Intel wrote in a white paper about its switch to SDN. “The network industry has been trending toward SDN over the last decade, and Intel Manufacturing has been deploying Cisco Application Centric Infrastructure (ACI) in factory on-premises data centers since 2018, gaining experience in the systems and allowing for more market maturity.”

Moving ACI to the manufacturing factories was the next step, and Colby cited Sanjay Krishen and Joe Sartini, both Intel regional managers, as instrumental in bringing SDN to Intel’s manufacturing floor.

The broad view of SDN in manufacturing

There are thousands of semiconductor companies globally, mostly in Taiwan. Yet the US Government CHIPS and Science Act of 2022 has incentivized more semiconductor manufacturing on US soil, and it is taking root.

“The use of cellular and WiFi connectivity on the factory floor has enabled these manufacturers to gain improved visibility, performance, output, and even maintenance,” says IDC analyst Paul Hughes.

“For any industry, software-defined networking brings additional scale and on-demand connectivity to what are now connected machines (industrial IoT),” Hughes says, adding that this also provides improved access to the cloud for data management, storage, analytics, and decision-making. “SDN allows networks to scale up securely when manufacturing activity scales and ensures that all the data generated by and used by machines and tools on the factory floor can move quickly across the network.”

As more semiconductor manufacturing springs up in the US, the use of SDN also “becomes one of the key steps in digital transformation where, in this case, a semiconductor manufacturer can collect, manage, and use data holistically from the factory floor to beyond the network edge,” says Hughes, whose most recent survey, IDC’s 2023 Future of Connectedness Sentiment, shows that 41% of manufacturers believe that the flexibility to add/change bandwidth capacity in near real-time is a top reason for SDN/SD-WAN investment.

The survey also showed that 31% of manufacturers say optimized WAN traffic for latency, jitter, and packet loss is another top reason for SDN/SD-WAN investment and is considered very important for managing factory floor equipment in real-time.

Intel has deployed SDN in roughly 15% of its factories to date and will continue to migrate existing Ethernet-based factories to SDN. For new implementations, Intel has chosen to use open source Ansible playbooks and scripts from GitHub to accelerate its move to SDN.

Intel certified Cisco’s ACI solution in time to deploy in high-volume factories built in Ireland and the US in 2022 and for more planned in Arizona, Ohio, New Mexico, Israel, Malaysia, Italy, and Germany in the coming years, according to the company.

Intel’s core partner on the SDN project is confident the benefits will continue to have a sizable benefit — even for a company of Intel’s size.

“The biggest benefit is that SDN helped Intel complete new factory network builds with 85% less headcount and weeks faster through the use of automated scripts,” says Carlos Rojas, a sales and business developer who worked on the project. “Automation and SDN enable better scalability and consistency of security and policy controls, and the ability to deploy micro-segmentation, improving Intel’s security posture and reducing attack surfaces.”

CIO 100, Manufacturing Industry, Networking, SDN

Supply chain disruptions have impacted businesses across all industries this year. To help ease the transport portion of that equation, Danish shipping giant Maersk is undertaking a transformation that provides a prime example of the power of computing at the edge.

Gavin Laybourne, global CIO of Maersk’s APM Terminals business, is embracing cutting-edge technologies to accelerate and fortify the global supply chain, working with technology giants to implement edge computing, private 5G networks, and thousands of IoT devices at its terminals to elevate the efficiency, quality, and visibility of the container ships Maersk uses to transport cargo across the oceans.

Laybourne, who is based in The Hague, Netherlands, oversees 67 terminals, which collectively handle roughly 15 million containers shipped from thousands of ports. He joined Maersk three years ago from the oil and gas industry and since then has been overseeing public and private clouds, applying data analytics to all processes, and preparing for what he calls the next-generation “smartport” based on a switch to edge computing in real-time processing.

“Edge provides processing of real-time computation — computer vision and real-time computation of algorithms for decision making,” Laybourne says. “I send data back to the cloud where I can afford a 5-10 millisecond delay of processing.”

Bringing computing power to the edge enables data to be analyzed in near real-time — a necessity in the supply chain — and that is not possible with the cloud alone, he says.

Laybourne has been working closely with Microsoft on the evolving edge infrastructure, which will be key in many industries requiring fast access to data, such as industrial and manufacturing sectors. Some in his company focus on moving the containers. Laybourne is one who moves the electrons.

Digitizing the port of the future

Maerk’s move to edge computing follows a major cloud migration performed just a few years ago. Most enterprises that shift to the cloud are likely to stay there, but Laybourne predicts many industrial conglomerates and manufacturers will follow Maersk to the edge.

“Two to three years ago, we put everything on the cloud, but what we’re doing now is different,” Laybourne says. “The cloud, for me, is not the North Star. We must have the edge. We need real-time instruction sets for machines [container handling equipment at container terminals in ports] and then we’ll use cloud technologies where the data is not time-sensitive.”

Laybourne’s IT team is working with Microsoft to move cloud data to the edge, where containers are removed from ships by automated cranes and transferred to predefined locations in the port. To date, Laybourne and his team have migrated about 40% of APM Terminals’ cloud data to the edge, with a target to hit 80% by the end of 2023 at all operated terminals.

As Laybourne sees it, the move positions Maersk to capitalize on a forthcoming sea change for the global supply chain, one that will be fueled by enhanced data analytics, improved connectivity via 5G/6G private networks, and satellite connectivity and industry standards to enable the interoperability between ports. To date, Maersk controls about 19% of the overall capacity in its market.

As part of Maersk’s edge infrastructure, container contents can be examined by myriad IoT sensors immediately upon arrival at the terminals. RFIDs can also be checked in promptly and entered into the manifest before being moved robotically to their temporary locations. In some terminals, such operations are still performed by people, with cargo recorded on paper and data not accessible in the cloud for hours or longer, Laybourne says.

Cybersecurity, of course, is another major initiative for Maersk, as is data interoperability. Laybourne represents the company on the Digital Container Shipping Association committee, which is creating interoperability standards “because our customers don’t want to deal with paper. They want to have a digital experience,” he says.

The work to digitize is well under way. Maersk uses real-time digital tools such as Track & Trace and Container Status Notifications, APIs, and Terminal Alerts to keep customers informed about cargo. Automated cranes and robotics have removed most of the dangerous, manual work done in the past, and have improved the company’s sustainability and decarbonization efforts, Laybourne notes.

“Robotic automation has been in play in this industry for many years,” he says, adding that the pandemic has shifted the mindset of business-as-usual to upskilling laborers and making the supply chain far more efficient.

“We have automated assets such as cranes and berth and then there’s [the challenge of] how to make them more autonomous. After the pandemic, customers are now starting to reconfigure their supply chains,” he says, adding that autonomous, next-generation robotics is a key goal. “If you think of the energy crisis, the Ukraine situation, inflation, and more, companies are coming to a new view of business continuity and future sustainability compliance.”

Top vendors such as Microsoft and Amazon are looking at edge computing use cases for all industries, not just transport and logistics. According to IDC, more than 50% of new IT infrastructure will be deployed at the edge in 2023.

Gartner calls implementations like Maersk’s the “cloud-out edge” model. “It is not as much about moving from the cloud to edge as it is about bringing the cloud capabilities closer to the end users,” says Sid Nag, vice president and analyst at Gartner. “This also allows for a much more pervasive and distributed model.”

Next-gen connectivity and AI on deck

Aside from its partnership with Microsoft on edge computing, Maersk is collaborating with Nokia and Verizon on building private 5G networks at its terminals and recently demonstrated a blueprint of its plans at the Verizon Innovation Center in Boston. The ongoing work is among the first steps toward a breakthrough in connectivity and security, Laybourne maintains.

“It’s technology that opens up a lot more in terms of its connectivity, and in some of our terminals, where we have mission-critical systems platforms, the latency that 5G can offer is fantastic,” he says, noting that it will allow the cargo to “call home” data every 10 milliseconds as opposed to weeks. “But the real breakthrough on 5G and LTE is that I can secure my own spectrum. I own that port — nobody else. That’s the real breakthrough.”

Garnter’s Nag agrees that private 5G and edge computing provide meaningful synergies. “Private 5G can guarantee high-speed connectivity and low latencies needed in industries where use cases usually involve the deployment of hundreds of IoT devices, which then in turn require inter connectivity between each other,” Nag says.

For Maersk, installing IoT sensors and devices is also revolutionizing terminal operations. In the past, the cargo in containers had to be inspected and recorded on paper. Looking forward, Laybourne says, the process will all be automated and data will be digitized quickly.

His data science team, for example, has written algorithms for computer vision devices that are installed within the container to get around-the-clock electronic eyes on the cargo and identify and possibly prevent damage or spoilage.

Edge computing with IoT sensors that incorporate computer vision and AI will also give customers what they’ve longed for some time, and most pointedly during the pandemic: almost instant access to cargo data upon arrival, as well as automated repairs or fixes.

“It can then decide whether there’s an intervention needed, such as maintenance or repair, and that information is released to the customer,” the CIO says, adding that cameras and data collection devices will be installed throughout terminals to monitor for anything, be it theft, lost cargo, or potentially unsafe conditions.

Maersk has also been working with AI pioneer Databricks to develop algorithms to make its IoT devices and automated processes smarter. The company’s data scientists have built machine learning models in-house to improve safety and identify cargo. Data scientists will some day up the ante with advanced models to make all processes autonomous.

And this, Laybourne maintains, is the holy grail: changing the character of the company and the industry.

“We’ve been a company with a culture of configurators. So now we’ve become a culture of builders,” the digital leader says. “We’re building a lot of the software ourselves.

This is where the data scientists sit and work on machine learning algorithms.”

For example, his data scientists are working on advanced ML models to handle exceptions or variations in data. They are also working on advanced planning and forecasting algorithms that will have an unprecedented impact on efficiencies. “Traditionally, this industry thinks about the next day,” the CIO says. “What we’re looking at actually is the next week, or the next three weeks.”

The core mission won’t change. But everything else will, he notes.

“We’re still going to have the job of lifting a box from a vessel into something else. Are we going to have autonomous floating containers and underseas hyperloops? I don’t think so,” Laybourne says, claiming the container industry is well behind others in its digital transformation but that is changing at lightning-fast speed. “Loading and unloading will still be part of the operation. But the technologies we put around it and in it will change everything.”

Cloud Computing, Edge Computing, Internet of Things, Supply Chain

For CIOs riding today’s rising wave of robotic process automation (RPA), leading-edge adopters whose mature implementations have paid off can provide invaluable lessons about how to make the best of the technology and where its use can lead.

Telecom titan AT&T is one such enterprise, having began RPA trials in 2015 to reduce repetitive tasks for its service delivery group, which had a large volume of circuits to add at the time, as well as various services in play for provisioning networks, says Mark Austin, vice president of data science at AT&T.

“These things would come in large batches, and they would have Excel files and people were literally typing these things in individually into the systems because they weren’t set up for batch,” Austin says. “We heard about RPA at the time, and we started trying it and all of a sudden we were able to automate one process and then the next process and it kind of grew from there.”

With the technology in its early days, the first thing AT&T IT did was go to its compliance and security experts for guidance on governing RPA, which helped the team make its automation tools stable and secure. The next step was to win the battle for hearts and minds within the company by turning skeptics into believers that automation could make employees’ lives better. Initial efforts focused on addressing unpopular, monotonous tasks such as order entry.

The pilots helped demonstrate how automation could fit into daily operations and workflows.

Within a year, AT&T had implemented 350 automation bots. More than six years into its RPA journey, AT&T has implemented more than 3,000 automation bots. Austin says RPA has helped AT&T recognize hundreds of millions of dollars in annualized value, saved 16.9 million minutes of manual effort each year, and shown a 20x return on investment.

Taking RPA to the next level

Mark Austin, vice president of data science, AT&T

AT&T

With RPA ingrained in its business process DNA, AT&T opted to combine automation with data science and the chief data office because it believes the future is in smarter bots that leverage AI functionality, such as OCR or natural language processing (NLP), an emerging strategy often referred to as intelligent automation.

“Tying those things together is pretty powerful,” says Austin, who runs AT&T’s data science, AI, and automation group.

By way of example, Austin points to what he considers one of the company’s biggest RPA successes: a bot his group has created that uses OCR to scan vehicle registration documents and NLP to understand those documents and any necessary actions AT&T must take in support of more than 10,000 technician vehicles, one of the largest vehicle fleets in the US. If payments are required, the bot can also trigger the payment process.

Being able to create automation bots such as these was invaluable when the COVID-19 pandemic first hit, Austin says.

“There were a lot of customers that were calling and saying they wanted to move the charges from this org to that org in their company,” Austin says. “Someone might call up and say they wanted to move 5,000 lines. What we do now is we have them interface with [interactive voice response (IVR)]. The IVR detects what they want to do and then it triggers a bot to send them a secure form to fill out. They fill out the form, submit that back, and we run the bot to automate the process to get it going.”

The company has also rolled out bots to help customers avoid overage charges. One such bot monitors usage of AT&T’s integrated voice, video, messaging, and meeting services, more than 21,000 records per minute, looking for overage charges above a pre-set amount. If it encounters one, it automatically notifies the customer and the assigned AT&T sales rep.

Codifying RPA best practices

After the first year of pilots, with demand for RPA spreading rapidly through the business, AT&T created an automation center of excellence (COE) to accelerate implementation.

“When you’re the size of AT&T, and you’ve had so many mergers and so many systems, there’s just lots of manual processes,” Austin says, explaining why it was essential to create a COE that could focus on implementing automation throughout the organization.

The centralized automation team now boasts 20 full-time employees and some contractors as well. Austin notes that the real secret to successfully scaling automation is spreading RPA knowledge throughout the organization. The COE helps develop, deploy, manage, measure, and enable automation projects across AT&T. More importantly, it seeks to educate subject matter experts in automating their own tasks and processes.

“Pretty early on, we figured out that if you really want to scale, you’ve got to move to training others how to do it, teach them how to fish, so to speak,” Austin says. “Ninety-two percent of everything we do with the 3,000 bots is done outside of my team. If you’re not an IT person, it’s maybe 40 hours of training.”

The company has trained more than 2,000 citizen RPA developers who have built the lion’s share of AT&T’s 3,000 automation bots. To support them, the company has created a “Bot Marketplace” where citizen developers can “shop” for ready-to-use tools and support to get their automation solutions up and running. The marketplace stores and shares low-code and no-code automation solutions and tools. It now adds roughly 75 new blueprints of reusable automation components every month.

As RPA knowledge has spread, Austin says the lines of business have started forming their own automation teams, creating a hybrid model in which the COE provides tools and support, while front-line teams in the lines of business implement automation.

“They even have some new job titles popping up,” Austin says. “We’ve got a couple process automation managers and automation developers that we’re seeing out there. On our team, we’re continuing to move to automate the process, the platform, and then tie in the data science side.”

When it comes to lessons learned, Austin has some advice for others out there who may be starting their RPA journey. First, start small and get some wins. Second, don’t try to keep things centralized. While the center of excellence has been essential to AT&T’s RPA journey, just as important has been democratizing the effort to scale the proliferation of automation within the company. Finally, evangelization is important. AT&T has created an internal automation summit where groups can present their automation projects to the rest of the company, show off their successes, and help spark new ideas.

Artificial Intelligence, Robotic Process Automation