Andy:
Thanks Jeff for being here. Everyone, this is Jeff with Avigilon. And if you don't know Avigilon, you should know Avigilon because they are a leader in physical security devices, mainly surveillance and access control. And they kind of wrote the book on marrying the two of them together, and we'll get into some of that here in a minute. But we're going to talk a little bit about AI today and what it is, what it means to you, and how you use it, and the benefits to your organization. So I'm going to turn this over to Jeff for the big topic of what is AI? How do we use it?
Jeff:
Yep. Andy, thank you for the intro. And as you pointed out, Avigilon leads the industry in the utilization and deployment of AI. And we are an end to end solution, not only with video and access control, and not just devices, but we also make our own software and firmware, so we're able to control the process. More importantly, we're able to make it easy to deploy and easy to use so you're not having to use third party overlays and bolt on pieces of software that you have to manage when version changes come out, and then you have to worry about if a version matches or is compatible with an upgrade. So that's one of the primary benefits that we were going for by going end to end is that it's easy for companies like yours to deploy.
Jeff:
And our roots were an ultra high definition video, and the kind of companies that were using us we're also using artificial intelligence, primarily object classification, where the camera and the AI are determining, are you a person or are you a vehicle? For the primary function of eliminating everything that's not those things, right? So when we use rules that say, if something enters this box, is it a person? Is it a car? Then we'll be able to ignore cats and dogs, empty bags or plastic bottles, or plastic cups that are rolling across the parking lot. The end result is that it allows you to search for things faster and when you create alarms, it has more meaning and you get more meaningful alert notifications.
Jeff:
So Avigilon got into the AI space through acquisitions, bought a handful of companies back between I think, 2012 and 2014. So we didn't invent it but we did take it to the next level. And again, our goal was to provide a much more friendly end-user experience and also create the end result of a more secure world, more secure workplaces schools, hospitals, government facilities, courthouses, and the like. And the biggest one being, or I should say, the biggest aspect of that at first was, like I said a minute ago, object classification. So the ability to know very quickly is that a person or is that a vehicle? From there, just let me take a second and kind of talk through, when people hear analytics and AI, there's a lot of things that are thrown out there that are, they probably fit the category, so let's just talk about it. There are other companies that you can draw green boxes and you can draw green trip lines and say, when something crosses this trip line going that direction or coming this direction, let me know.
Jeff:
Or you can draw a green box and say, if something enters that green box let me know, create an alarm that sends an email or pop something up on a screen. And that is a great start. But the problem with that is that you get false alarms that are created by raindrops on heavy rain days. You get false alarms that are created by clouds that are casting a shadow on the ground, especially on a very bright, sunny day, a shadow coming across. So because cameras see motion as changes in the pixel, actually they see motions changes in the gray scale.
Jeff:
So where a flickering light can change the degree of shading of that wall is white right there could change how white it appears to the camera. And so I bring that up to make the point that you can have green boxes and green lines on the ground and still get a lot of stuff that you don't want to see. Whether it's empty water bottles blown across the parking lot by a breeze, tree branches that maybe you don't want to mask them out because it's kind of near a door that you want to watch. And so with object classification, just kind of circling back to kind of reiterate the point, what you're doing is able to say, that's a person, that's a car.
Jeff:
I want to see everything that enters that green box that's only a person and ignore cars and everything else. Or I want to see anytime a car comes down this road going the wrong direction. So we start with object classification as the basis, and then we employ rules. So drawing a box and saying, show me when something enters this area, object enters or object appears. Another rule might be object leaves in area. So if you have a reception area where there's a security guard, that has to always be there and when they leave, somebody else has supposed to be there. The camera can tell you when there's nobody there, with a rule that says object leaves area. And of course you can program that to, for more than 30 seconds if there's nobody in this area. And then you check that you want it to look for humans in that area, right? People. So it's making a much more meaningful alert notification.
Jeff:
Other things that get talked about in the world of analytics are a license plate recognition, facial recognition. And both of those are, those are absolutely analytics, not really what we would consider full-blown artificial intelligence, but it does fit under the category of intelligent video. And I think that's something that needs to be talked about a whole lot more because the quality of cameras has just gotten so good that it's not difficult to find a camera that does a decent job of recording an image. But where we are now is intelligent video, it has come, it is its time, right, for intelligent video, where people are spending money on cameras without thinking about or being educated about intelligent video and the ways that it can help them. So where people years ago were happy with just having cameras, recording scenes. Now your video system can be pointing things out to you proactively so that you can be heads up before something turns into an incident, right? So getting back to the different kinds of analytics…
Andy:
Let's chase that for a little bit though. Like the phase detection, there's some great uses for that, just with matching people that are coming in and out with the appropriate card swipes too, if it's integrated. There's also the bigger advantage in using face detection, you could actually grab… Say it's a school, you have a disgruntled parent, you could flag them in the system via the Facebook picture and have that camera alert you when they step on campus.
Jeff:
Absolutely, so facial recognition, I think really kind of burst onto the scene, like you mentioned, as a tool that was used in conjunction with access control. It was a form of a credential. So where you have the three main credentials, what you have, what you know, and what you are, right? So what you have as a card or a key fab, what you know is a four or a six or eight digit pin code. And then what you are is either your hand or your finger or your thumb or your face, or there's even stuff that looks into your eye, right? The AI has gotten better, cameras have gotten better, and facial recognition has gotten better to now it is able to be used to, just like you said, point out when somebody of interest comes onto your property.
Jeff:
The facial recognition that my company sells as part of our offering, you can have multiple lists. So you can have a known bad guy list. These people are bad actors and the minute they step foot on my property, I need to know about it immediately and respond this way. You can have your executives and VIP list of people that, very important people that come on my property when she comes on, or he comes on, let me know about it so that we can roll out the red carpet. Then there's that in-between stuff where maybe it's a school and there's parents going through a custody battle, maybe somebody is not supposed to be picking up the children, you're not going to roll cops and lights and sirens, but you want to be heads up so that you can kind of have somebody come out and meet that person, right, so that's maybe a different list.
Jeff:
And just as you aptly pointed out, you can capture their likeness off of LinkedIn, Facebook, whatever social media, and upload it to the file to create that watch list even if they've never stepped foot on your property. If somebody threatens the owner of your company or says, I'm going to go and do some damage at this building, because I don't believe what they believe, you can put them on the watch list, even though they've never been on your property. So that is another great example of where we are with intelligent video and the use of facial recognition. Similarly, license plate recognition, certainly not new. There have been a lot of developments that it's an improvements over the years where you used to have a camera down really low and it had to be looking straight up the tailpipe of a car to get a good plate read. Now with the proper camera and a proper lens, you can achieve the pixel density that you need for an accurate plate reader and angles aren't as critical as they used to you.
Jeff:
You still don't want to be looking for license plates off to the side or way up super high, but it's just not as challenging as it was say 10 or 15 years ago. So people are looking at and saying on these two driveways, I want to know who's leaving my property so that if I want to, I can respond accordingly. License plate recognition of course is that software feature that lets you do two primary things, right? You can type in the known plate number and search for it. You can type in a partial number for instance. So if you came up to me and said, Jeff, I saw somebody in the parking lot doing something bad. And they drove off in a red car that had a XYZ and I can't remember the rest of the plate, we could type in XYZ and it would show us everything that comes close. And then you look at the resulting video clips and say, "Hey, that's the one right there."
Jeff:
Now you have a full description. It's a red Toyota Camry. The full plate is AFGXYZ3, right? So that's the full plate, but you only started with XYZ. The other ability that you're able to do is put that plate on a watch list. And I tell people, if you don't need to put a plate on a watch list or search for it with the known digits, then maybe LPR isn't what you're thinking. Maybe you just want license plate capture, a camera that's just getting a good capture of those plates. When, for those times that you need to go back and see what the plate was, because there's this a little bit of cost to license plate recognition. But more and more people are using it, prices have come down, and it is seen as a very useful tool for people for various different reasons. Sorry, I kind of talked a little bit flowing there.
Andy:
No, that's fine. And there is a difference between license plate capture and license plate recognition. I may yank this out as an…
Jeff:
Sure, yeah. Where license plate capture is just getting a good visible capture on the camera through the deployment of the right camera, the right lens, the proper placement, making sure that the lighting is correct.
Andy:
And then the actual LPR license plate recognition is actually being able to search for a plate.
Jeff:
Right?.It's it's software that allows you to put a plate on a watch list or go in and type in a plate and search the video over numerous days and see how many times they've been on your property or how many times they've driven around your building.
Andy:
Great. I think the next thing we need to talk about in this is analytics, people counting, and you guys have this really cool feature in your system to where if you find a person and they stole something, say at the mall, you could go through, highlight that person, and it'll go back through and basically build a timeline, finding him coming in and going back out.
Jeff:
Sure. Yeah, you're talking about the appearance search, which is really taking intelligent video to the next step. It's what we, at our company, would consider a true application of artificial intelligence and where we had our roots in machine learning and now we're using convolutional neural networks. So similar to how our brains work it's layered comparison of known things where in the old days you had to present image, image, image, image, this is the thing, this is the thing, this is the thing, with data points on it. Where with convolutional neural networks, it's a layered approach of comparison. Well, it has wheels, but it's too to be a vehicle. It's large enough to be a vehicle, but it doesn't look like a truck. That's kind of a rough example of what we're talking about with that convolutional neural networks.
Jeff:
And so kind of getting back to what you were talking about, the appearance search is really a game changer. And at first, when we rolled out with it in the beginning of 2017, it was for companies and customers that had numerous, numerous cameras. North of 25, 30 cameras, ideally if they had 84 to hundreds of cameras. And you would find a target of interest or a person of interest, and there would be a bounding box around that person. And in the video screen, without having to go to some special search screen, you just left click and then a dialog box says, find appearances of this moving forward or going backwards. And so if you have somebody running around your shopping mall or running around your school with a weapon or acting crazy, you want to know where they are right now. So moving forward, or if you're doing a forensic investigation and you start with a video clip of that person as they enter the lobby and you want to see everywhere else that they went, then you would move forward but you can also go back.
Jeff:
So hospitals, sometimes, they're concerned with what are called creepers, right? And those are people that go from room to room and maybe looking in people's bags and people's purses. And when they find somebody that's doing that, they want to know where else that person has been. So in that use case, you would click find appearances instances of this person going back, how did they get in the building and everywhere that they've been. And then, as you're looking at these tiles, what it does is it presents tiles of people that it thinks are targets. And at first it may be not exactly the person that you're looking for. There's a graph on the bottom where the computer judges, how well it thinks it matches what you're looking for with spikes and valleys. And you can grab a box and move it over those spikes and valleys, and it'll change the tiles and you can very quickly see, even if it's not on the first go, who your person is, just by moving it two or three times, you can very quickly look at and scan numerous cameras, numerous periods of time and find that person.
Jeff:
When you click on it, the AI goes, Oh, this guy, and then presents more of them over time, over multiple cameras. And to the point that you made, it helps with investigations because you're able to go in and put stars on the ones that are of interest. Hey, this is him entering this room, this is him picking up something in the hallway, this is him coming in the door when he first got here and it puts those starred results on the timeline that you can then export. So if you're a forensic investigator, or if your customer as a forensic investigator, this is going to help make things super, super easy. That was when it first came out, you had to start by clicking on the person.
Jeff:
About a year and a half later, in the summer of 2018, we came out with a newer version of that software, 6.10, which allowed you to do appearances by description. So somebody could come in and say, "Andy, I was just accosted by this guy out in the parking lot. He was wearing black pants and he had on a blue shirt and he had brown hair, and you could type in those, you could choose male, brown hair, blue shirt, black pants. And you could start from that where you have no idea which camera it is, and it will show you results, right? So always improving the AI, always making it easier for you to utilize and making it easier for your customers so that they can actually solve their workplace problems, whether it's finding a bad actor or one of the other features that we have, when you have our access control connected with the video, is that you can do what's called an identity search.
Jeff:
And if you need to see all of the places where Andy was, using his card, you can start with typing into the access control, hey, I'm looking for Andy. It shows you all the Andys in the organization, this is the one I'm looking for. And then it shows you a timeline with the associated video of the cameras at those doors. From there, you jump off and do the appearance search and everything I just described over the last couple of minutes, it starts with the access control. You're in the video interface, but you jump into the access control to see which doors he used or she used, and then you're able to do, just like I said on the appearance search, go forward or go back. And if you're doing an identity search, many times, people want to see how did Andy get this card because he's not using his own card? Did he pick it up off of Jeff's desk or did Jeff give it to him? Do I need to fire Andy or do I need to write them both up for violating company policy?
Jeff:
You can go back and find that moment where you see Andy coming by my desk and opening my drawer and taking my card out. Or you find that moment where we're at the water cooler in the lobby and I hand you my card and say, "Hey go around and act like you're me because I want to take a couple of days and go fishing." So that's kind of the use case of the identity search, kind of taking the AI capabilities and merging it with what we have in our access control suite and bringing it together, make it very powerful.
Jeff:
Now getting into the times that we're in now. So we're recording this, this is November the 18th, 2020 where COVID-19 is a right? From that, we have leveraged that into contact tracing, which is an important tool where people are trying to figure out if somebody in their organization comes down with the coronavirus, they want to see where they've been, what they've touched, who else they might've come in contact with and just doing a search in the access control alone may not provide that for you. So again, a very powerful marriage between the access and video using intelligent video and the things that we're able to do in the access control as a jumping off point.
Andy:
Great. People Counting?
Jeff:
People counting. For many years, it was something that we shied away from, and it was more a business analytic where you saw big box stores that wanted to see how many people were coming in, how many people were going through the checkout line and that kind of stuff. And again, due to the times that we're in, we saw the usefulness of being able to count people because there are rules, and in some states, laws that limit the occupancy to 50% or 67% of what the maximum occupancy of what that building or that room could be, it is now less. Classic case in point, it was a true conversation I had several months ago at a courthouse where I asked the captain, I said, "Are your courtrooms limited to the occupancy like to half?" And he said, "Yes, they are." And I said, "How are you doing that right now?" He said, "The bailiffs have to manually count who comes and goes and they have to make sure that nobody else comes in. And I'm currently sometimes using two bailiffs to do that, one at the door, and one inside the courtroom."
Jeff:
I asked him, "If a camera could do that for you and display on either a tablet or a monitor, whether it was a go or no-go, would that be helpful?" He said, "Absolutely." And so that is now helping that sale progress because it's solving a very real problem for that courthouse and it's reducing the manpower requirements and sometimes they're not able to go out and hire more people, they just need them to work overtime. So by making that go away, we're actually saving them money, using the people counting for the COVID situation that we're facing.
Andy:
Great. There's one other cool thing about the intelligent video, I guess if we want to say, the analytics, the AI, how it can actually help, through what Avigilon has in their honeycomb. We were chatting with some friends of ours and they take care of the people side of security. They do training as far as training security guards and that kind of stuff, we're going to try and get them on to do a podcast as well. But when they go onsite to these places, everybody's like, "Hey, come check out our NOC, come check out our watch room." And they get in there and there's like 100 three inch squares on four or five TVs. And no one's really watching anything, I mean, no one can watch 200 images at the same time. You guys have your honeycomb, which this is really cool because what it does, and you can elaborate a little bit on this, is it will actually intelligently flag cells based upon analytics in the video. So now the security guard is not sitting there at 2:00 AM, kind of awake, kind of not. All of a sudden now he has this red thing flashing in front of him going, Hey, there's something at this zone kind of thing.
Jeff:
Sure. Yeah, the honeycomb that you're talking about is what we refer to as the focus of attention interface, and we rolled it out with version seven that came out in the early summer of 2019. And it stems from our relationships with our end users who will invite us into, not only do training, but look at how they're using the system and make suggestions. And it becomes a two way dialogue where they suggest things to us like, this would be really cool if. And like I said in the beginning, our company was focused on ultra high definition. Many of the companies that were using it were also using these third-party analytic products. And we saw it as something that would be easier if we could make that a part of what we ship, right? So our cameras that have analytics are self-learning, you just put them up on the wall or on the outside of the building.
Jeff:
So fast forward over the last seven or so years that we've been doing that very successfully with some Fortune 500, even Fortune 100 companies, very big names that would say, "It's really cool that we have these alert notifications coming in, but it's a challenge that I have all these monitors, large," prices have come down, people have 70 and 80 inch monitors, but there are some command centers where it takes two and three operators watching that. And what we've learned is that the human brain just cannot handle that. Our Chief Technology Officer, Dr. Mahesh, refers to it as the scarcity of human attention bandwidth. And so you can sit there and be looking at a very large grid on a monitor that's very large. So I'm not talking about not being able to see the cameras because you have so many that are up there that the tiles are too small. You can have them be, each tile is the size of an 8.5 x 11 piece of paper, you can see clearly what's in it, your brain just can't process that.
Jeff:
So what they've noticed, they've done studies on how long you can look at a wall of video before you start missing things and it is exponentially shortened by the cameras that you add. So you go from one to four to 12, and it becomes very difficult for your officers and your loss prevention personnel to see what's going on. And so through our studies and through the feedback that we get from end-users, we heard that that was a problem. And we came out with a focus of attention interface, the honeycomb that you speak of, where it takes all of your cameras, and you can organize them into groupings where you have all of your indoor cameras on the first floor, all of your outdoor cameras, all of your indoor cameras for the second floor, or you can group them East wing, West wing, you can group them however you want, but typically, people will put them in an area where if an officer or a guard sees something on that honeycomb very easily, they can mouse over the surrounding cameras.
Jeff:
And I need to stop for a second and talk about how we use color coding. So an empty honeycomb on that focus of attention will indicate that the camera is no longer connected, right? You have a camera outage, which some larger companies, part of their protocol, part of their SOP is at shift change, go check all the cameras to make sure they're online. Well, if you have a visual representation, how much easier is that? If you could just look up at the monitor and go, "Hey, I have two empty boxes. Let's see which cameras those are," and give the integrator a call so they can come fix them. A gray hexagon means that the camera's online and there's nothing of interest happening, so that's a good to go, right, nothing going on, but nothing that I want to see.
Jeff:
Blue means motion. It could be any kind of motion could be a cat walking through it could be raindrops on a heavy rain day, it could be branches from a tree that are moving in the scene, it could literally be anything. Where teal comes in, a blue border with a teal center, now that's going to be unusual motion detection, where the camera's learning the pattern of life over, say two weeks or more, where it says, hey, traffic running through this seen as normal, cars doing donuts is not. Or people in my warehouse is normal, but a person climbing the racking is not.
Jeff:
So that it's going to point out to you, hey, you've got an OSHA violation. Hopefully you catch it and hopefully the person doesn't fall and crack their skull, right? So the camera's learning, what's normal and what's not normal. And our software does that on our analytic and our non analytic cameras. And by that, I mean, the ones that don't have a built in self-learning analytics with object classification. The other thing that teal box means that there's an object, a classified object that has entered the scene, a vehicle or a person. So when you see that teal box, it's either a person or an unusual motion event or a vehicle. And then you have yellow where that's something that you really kind of need to pay attention to because it's either a vehicle has come through on the LPR, the license plate recognition watch list, or a person has walked past a camera that's on the facial recognition watch list.
Jeff:
And then the last one would be red, where you've actually created an alarm that says, if this happens, this is an alarm and I want to know about it at the highest level of priority. You can set alarm priorities, you can make it where people have to create notes before they can clear the alarm. There are a lot of different things. All of those show up on that focus of attention interface. And as you mouse over those hexes, you actually can see what that camera sees. You can double click it and have that camera come up. It's going to make that focus of attention, all those little hexagons go down to the lower right-hand corner. So your guards, your operating people are still able to have situational awareness, but they can examine forensically what's happening in the scene on their larger screen here and down in that lower corner if another hexagon pops that's red or yellow, they can know that, Hey, I need to hurry up and resolve this one or maybe I need to pass it off to another operator to my left or my right.
Jeff:
So when we first came out with that focus of attention interface, the goal was to make video walls go away. And what we found was, people still probably like seeing cameras and utilize the focus of attention interface in conjunction with it because it helps bring things to your attention. Great story I love to tell actually was just not that very long ago, and I was at a hospital. They had recently upgraded their software to version seven, and I went with the integrator to go do some training for their operators. And the very first shift that we were sitting with, I said, "Hey, you know that you have this focus of attention as part of what your integrator recently installed. Have you ever played with that?" And they said, "No."
Jeff:
Now I need to stop and describe this command center. Directly in front of the tables or the desks were four 60 inch monitors, and then the walls angled, and there were probably six, 20 or 32 inch monitors off on the sides looking at over 140 cameras. And I showed this one lady the focus of attention interface and she said, "Oh man, we need to do that right now. It is so difficult for me to pay attention to all these cameras, and I'm on night shift," because we were doing the training at like 4:00 in the afternoon. And she said, "We need to assign this to one of the monitors right now and you need to make sure that you talk about this on all the other shifts and share it with the leadership."
Jeff:
And so over the next day and a half, where we did training with multiple shifts, we showed them that shared it with the leadership, and that's now part of what they do, because looking at all those monitors was just so difficult, where now they can actually have the AI and there's some non AI things that are happening there too. So alarms coming in from access control, alarms coming in from a dry contact on a generator or a piece of machinery. There's a lot of different things that can push to that to bring heads up awareness to that person who's having to sit there for an eight hour shift or a six hour shift and pay attention to what other circumstances would be an overwhelming amount of data coming at them.
Andy:
That is great.
Jeff:
That was the long description, sorry.
Andy:
That's great. It's good stuff, and it's stuff that people need to start thinking about when they're upgrading or installing or whatever, even just partially upgrading. This is a new technology that can save… It can save lives, it can save insurance claims, it can save a lot and it will probably pay for itself.
Jeff:
It absolutely pays for itself. Kind of an analogous story is restaurants and hotels pay a lot of attention to their lighting. When lights go out, it's a top priority to get lights replaced so that the lighting is good because studies have shown that when there's poor lighting, it affects morale and it affects people, you don't want customers having a less than positive experience. And similarly in the workplace, when employees are faced with challenging situations that they have to overcome in the normal performance of their duties, it wears on them. And when you give them tools that help make them more effective and more efficient so that they can solve the problems they need to for you as their employer in a manner that's easy, they can produce improved results, you're happy, the people that they work with are happy, the internal customers, the employees feel safer and it really drives morale and helps make a more productive workplace. And your customers that are coming into those workplaces feel that.
Jeff:
So I've had conversations where people say, "Jeff, I've just spent the last two days scrubbing video, I wish that you had been here last week. I mean, having the tools that you've shown me, being able to do a search for just a person and not have to look at everything would have saved me the last two days of looking for what I was trying to find."
Andy:
Yeah. I mean, it's time-saving and it's important, people need to think about this stuff. So I want to thank you for your time today.
Jeff:
Absolutely. My pleasure. Thank you for having me.
Andy:
It was great, it was great. Thank you for coming. If we can help you with any of the technology in your building, whether it's surveillance, access control, security, AAV, networking, head over to our website, scroll down and fill out that contact form, we'll be happy to answer any questions, make some suggestions for ways to better improve the technology in your space.