Integration — I

6 min readOct 8, 2021

integration noun

the act or process of combining two or more things so that they work together
the action or process of combining two or more things in an effective way

Conversation

If we want to call something a conversation, there has to be two-way communication there.

In our daily life, we have lots of solutions to make complete conversation. I’m going to share some of them which are also related to system integration.

If the two people don’t know any common language to talk then they use TRANSLATOR to solve their communication problem to understand each other.
Sometimes people take notes of their conversations to not forget important details. They might use that notes to share with other people who are not directly in conversation.
People also might lie in their conversations. So the other people get wrong information as a result of that INCONSISTENT situations can occur. Sometimes we can understand immediately there is a lie or sometimes it takes longer to realize and sometimes we never realize.

These are some of the real-life examples of our conversations. I just wanted to mention that real-life conversations are very similar to system integration.

Conversation Patterns

Before we start our system integration, we need to define our conversation partners and their needs. In my experience, this is the most important and time-consuming part of the integration. According to Gregor Hohpe who is one of the writers of Enterprise Integration Patterns, he calls this first step of integration is A-Discovery. (he is using this term more at the system level but we are going to use the same term for both system-level and organizational level.)

So let’s create the environment that we’re going to use in this article series and then discover it together.

Our integration is going to be between internal and external systems.

Internal is our organization and external is some other environment(SAP, Dynamics 365, Salesforce, etc.) outside of our organization. Let’s define our processes while integrating these systems.

Generally, external systems have a common language and standard for their all clients. However internal systems have their own language and need different from other clients of external systems.
At this point we need to answer these questions;
Are we going to develop a new flow out of standardization of external systems? Are we going to use standardization of external?
or both?
Even in both cases, we have to define our business needs. So before getting into system design details let’s define our organizational needs.

As you can see from the diagram above, our X, Y teams are going to use standardization of external systems. While using standardization of external systems, still there is a problem with translation because these internal and external systems languages are different.
So at least one member from each team should analyze the integration flow between internal and external systems. (We might figure out something else but in this environment, every domain has to integrate its own flow.) For our Z, T, W teams that are not going to use standardization of external systems this is more difficult because these both internal and external systems need to develop new flows. So what I would like to mention with this example is before systems communication we need to define our real-life communicators to shape our integration.

So let’s define our A1-communication structure.

Here is our communication structure. There can be several different ways to build communication structures. What we have done is in here, we created a team that understands both sides' integration needs and encapsulates them. These two-way red arrows are on the diagram mean handshakes. So this is the place late stage of discovery and early stage of B-Starting a Conversation.

So let’s have a look at B1-Handshakes/Consensus (mostly business side uses Handshakes term and engineering side uses Consensus term so we are going to follow the term Consensus)

Definition of Consensus

The process by which we reach agreement over system state between unreliable machines connected by asynchronous networks. (Heidi Howard)

I was thinking of a real-world example for this scenario and Facebook disappeared from the internet for a while on October 4th, 2021.

Facebook published that blog post after they find the root cause then they informed us. I underlined the keywords that related to this article. So if you are Facebook the dependency network is huge and if you also use your services internally as Facebook your internal systems are also effected by communication interruption. Yes it was about communication and that system problem became a business problem, people problem and If you are Facebook it’s an internet(world) problem.

If you are interested, you can read more about the details here and here.

As the Internet, our systems are need to communicate properly.
How Internet solves this problem?

Today’s events are a gentle reminder that the Internet is a very complex and interdependent system of millions of systems and protocols working together. That trust, standardization, and cooperation between entities are at the center of making it work for almost five billion active users worldwide.
cloudflare

B1.a TRUST

First of all, it’s not system security that we are going to talk about under this title. We are going to discuss how we can achieve trust.

We need to know that trust is dynamic. Because trust is dynamic we need to validate trust periodically to be sure could we still trust? But what if we don’t have “time” to validate? At that point, we need to have initial points.

Again we can use our real-life examples to describe initial points;

Reference, Recommendation: If we have a source that we already trust, we can use the source’s validations that the source already experienced. This can be a tool in our system design or this can be another system that we already use in our architecture. So the main thing is if we have trustworthy sources we can use their experiences.

Reputation: For example, Azure, AWS Cloud, and other cloud providers have their own strengths and weaknesses. How we decide is we look at their reputations right? We benchmark their services, compare their services and then we decide. It’s also an important part of our integrations.

Experience: Our systems mostly have legacy systems or we have past experiences from different companies and we gain experiences on architectures, design principles, problem solutions, mistakes, etc. Once we got that experiences we are able to think about fail scenarios, fall-back scenarios, corner cases, etc. So this experience makes our systems more resilient. As Facebook said in their outage report.

So if all these are not possible then you need to build trust from zero. :)

As a result of the first part of the “Integration” article series. I wanted to mention that integration is not just a technical thing. There is a lot behind the scene of technical decisions. Also, social experiences that we gain in our social lives. In the next part of this article, we’ll continue to discuss integration in a way technical and sociological concepts.

P.S. Integration II will be about Consistency.