Augmented Reality proof of concept allows audio recordings to visually float in 3D space

Share on FacebookTweet about this on TwitterShare on Google+

With all the audio analysis and enhancements that we do, I am often looking at audio waveforms and spectrograms. That’s one important way we explore, measure, and judge how audio filters are impacting the original audio.

Below are examples of how a voice is seen in today’s software on a waveform (the blue signal), and a spectrogram (in orange). Both are in stereo.

analyzing audio from a voice

Waveform analysis of spoken voice

Now, a revolutionary visualization of audio signals might soon help us with our forensic audio work.

Watch this amazing new test concept using the Apple ARKit (Augmented Reality Kit). The ARKit is software included with all iPhones and iPads.

This concept test allows the user to create audio that floats in space which the user can walk through to explore the sound signal.

With the proper programming, audio filters, and acceptance in the forensics community, I can see this being the future of audio enhancement, analysis, and editing for court situations.

Posted in Enhancement of audio and video | Leave a comment

Create Your Own Video Evidence and Help Solve Crimes

Share on FacebookTweet about this on TwitterShare on Google+

Create Your Own Video Evidence and Help Solve Crimes

video evidence

Your video evidence may solve a crime.

Under normal circumstances, the police are responsible for finding video evidence, or any evidence for that matter, of a crime. The police will then process and analyze the video evidence to solve the crime and prove their case. But what if everyday citizens could have an opportunity to create video evidence that couil help solve crimes? This is more or less happening in Livermore, California.

As of 2015, the Livermore Police Department has created a database of privately owned surveillance systems so that they may determine who is in possession of potentially surveillance video systems that may have recorded useful video evidence of a crime. The police are requesting that anyone with surveillance register their system and its location with the department. If the police have reason to believe someone’s registered surveillance system may have recorded a crime, they will request temporary access to the owner’s surveillance system so that the video evidence may be extracted and analyzed by police.

Citizens On Patrol

For the police to access privately owned surveillance may increase video evidence for prosecuting criminals. However, police accessing private citizens’ surveillance may also be another step towards an invasion of privacy via 4th Amendment violations. So how is the public kept safe from police overstepping their bounds?

First, the Livermore police are not asking for permanent access or for a live stream of these private surveillance systems. Rather, the police plan to only access surveillance systems on an as-needed basis. Further, individuals who have registered their devices are under no obligation to give police access to their systems, even if police suspect a crime has been captured. Police promise to request access and only then recover pertinent video evidence.

If the Livermore Police Department’s surveillance program is successful, this may be a positive step in solving crimes faster and more efficiently. Advances in surveillance video cameras, data storage (such as cloud-based storage), and the internet have made it possible for the police to do more than what they’re attempting in Livermore, California.

Homemade Surveillance Video Evidence to Solve Crimes

Most new privately owned surveillance systems are designed to be accessed from anywhere in the world via the internet. In other words, if a user has a computer or smartphone with access to the internet, they can see a live stream from their surveillance, or they can access saved video evidence from their cameras. Advances in all this technology is such that multiple surveillance systems may be viewed from a single access point. For the police to do this would give them unprecedented access to live surveillance feeds, thus allowing them to catch crimes being committed in the act. Such efforts may be able to decrease response and investigation times, as well as help to prevent crimes from happening.

Posted in Enhancement of audio and video, In The News - Surveillance | Tagged , , , , , , , , , , , , , , , | Leave a comment

Teaching an MCLE tomorrow in Fullerton area

Share on FacebookTweet about this on TwitterShare on Google+

We’re looking forward to teaching attorneys about the newest techniques in video, audio, and still image forensics. All examples and material taken from our actual cases.

I expect a great crowd tomorrow, so join us!
Here are the details!

Posted in Enhancement of audio and video | Leave a comment

Why Snapchats Aren’t as Snappy As They Say

Share on FacebookTweet about this on TwitterShare on Google+

Why Snapchats Aren’t as Snappy As They Say

snapchat photo evidence

Snapchat hidden files may still be useful photo evidence

Photo evidence can often come from some very unexpected sources. Snapchat is still a highly popular app right now, especially among teens and preteens. If you are not a teen or preteen, you may need some explaining. Simply put, Snapchat is a picture-sending social media app; but what makes Snapchat different, however, is that the pictures (typically referred to as “Snapchats”) taken and sent via the Snapchat app will automatically self-destruct after a few seconds and are never seen again. Supposedly. (Unless someone screenshots them.)

What does that mean for you or a potential court case? With Snapchat’s popularity, it’s highly probable that Snapchat may become an important part of a court case. Depending on the nature of the case, the Snapchat generated photos -which have “disappeared”- may become valuable photo evidence. So then what about the photos that supposedly disappeared? Well, they don’t go away as quickly as people think.

For instance, did you know you can recover Snapchats from a phone, even after they’re deleted?

Snapchats Don’t Go Poof

snapchat photo evidenceThat’s right. Snapchats don’t completely disappear. As Richard Hickman explained to KSL.com, the pictures are just given the file extension “.NOMEDIA” by the Snapchat app, which makes them invisible to most normal phone users. Now, eventually, those files will be actually wiped from the phones storage in order to make more space, but they often hang around for quite a while, and what’s more is that those “disappeared” Snapchats can be recovered from a phone.

Snapchats Are Recoverable

If you were to Google “recover snapchat photos android”(or iPhone), you’ll find many articles or downloadable programs on the topic of recovering these hidden files. Recovering the pictures properly, however, will require a forensic expert who has the right programs and knows what they’re doing. This is especially true if the pictures may be used as evidence in a case.

The Point of Snapchats Photo Evidence in Criminal Cases

snapchat photo evidenceBut why should anyone care about old Snapchat photos? Well, these Snapchats could be material evidence in a case. Cases such as a child pornography case, a family law case or child custody case, or even a civil case such as a slip and fall / personal injury case. The relevant point is that most people don’t think of Snapchats as hanging around, since the app automatically deletes the images. So someone who used Snapchat to capture evidence, but then assumed that evidence is safely deleted are probably not as likely to think they need to go back and erase that evidence. Having a forensic expert recover that hidden evidence, however, could work to your case’s advantage.

NCAVF and David Notowitz attended a photo evidence forensic symposium and heard forensic expert Joseph L. White, MS speak about a case of a man who tried to claim his Snapchats didn’t contain underage nudity. (Which can be found in the AAFS reference library.) The suspect in this case assumed he was safe as the Snapchat images had been “deleted”. A recovery of these images, however, revealed that they did in fact contain underage nudity, which was discovered through the use of forensic video analysis and software. These recovered images were then used as material evidence in the case.

Conclusion

When dealing with a case that involves images that may have been taken with a cell phone, it’s important to be aware that Snapchat may have been used to capture images. And even if Snapchat deleted these images, it might mean there is still hidden evidence buried on the phone’s storage. Therefore, don’t dismay if you need a Snapchat back. It is frequently possible to recover Snapchats from a phone to use in a case. All you’ll need is a little bit of digging, and a forensic video specialist, to bring them back from the grave.

Posted in Advice for attorneys and police on recent surveillance issues | Tagged , , , , , , , , , , , , , , , , , | Leave a comment

What Should Your Image and Video Forensics Expert Know?

Share on FacebookTweet about this on TwitterShare on Google+

What Should Your Image and Video Forensics Expert Know?

Video Forensics expert

Suppose you have a case that has quite a bit of video evidence or still image evidence. You know it needs to be looked through. Or you know that an image or two will be pivotal depending on what you find out from them. The prudent thing to do would be to decide that you need a video forensics expert. But you’ve never hired one. What should an attorney be looking for in a video forensics expert?

According to the Scientific Working Group: Imaging Technology’s (SWGIT) “Best Practices for Forensic Video Analysis,” a video forensics expert should do the following:

Understand what the work entails within the forensic environment

Basically, your video forensics expert needs to know his job. He can’t just be some programmer who knows a bit about file structure. Or a video editor doing forensics on the side. He should know what needs to be done in the context of a forensic environment.

Know the subject

If you’re going to hire someone, make sure he has knowledge about your particular problem. Has he worked cases with surveillance videos before? Does he understand about violent crime? Make sure he understands the case you’re handling. Make sure your forensics expert also understands the defense or prosecution strategy you’re going to use to prove your case (or disprove the other side’s case) so that he knows the aspects of evidence to look for as he works on the video enhancement.

Know the right video and image forensic techniques

Of course, the expert may need programming, software, and video expertise as well. They shouldn’t just be a glorified private eye using internet tutorials.

Know Forensic GVideo Forensics expert best practicesuidelines (SWGDE)

There are several organizations that recommend various forensic guidelines that may be followed when evaluating or enhancing evidence. The Scientific Working Group on Digital Evidence (SWGDE) is one such group. SWGDE publishes forensic guidelines that are studied and observed by many forensic experts. SWGDE is one of many organizations that contribute to a variety of standards in acceptable forensics techniques and practices for video forensics.

Understand the “legal precedent for the use of specific image and/or video processing techniques”

Your video forensics person should know the lay of the legal land; otherwise, they might make mistakes that would keep crucial evidence out of court.

Understand and use “appropriate casework documentation”

For a similar reason, your forensics person should know how to keep correct documentation. These are just several of the guidelines given. You can find more here. Basically, you just want to make sure that the person you hire is careful, precise, and knows their stuff.

Posted in Enhancement of audio and video | Tagged , , , , , , , , , , , , , , | Leave a comment

Can You Spot All the Fakes? Images, That Is

Share on FacebookTweet about this on TwitterShare on Google+

Can You Spot All the Fakes? Images, That Is

Video Forensics
Not sure whether a key image or video in your case is faked or original? Photoshopped images and video have become a big problem, as editing programs get more sophisticated and easier to use. Just a couple of clicks in Photoshop and you can make almost anything disappear. But what if your case has a lot of images that you can’t tell are false or real. Can you sort through lots of photos for fakes?

Right Now

Nowadays, you can get programs that will at least attempt to tell you if a picture has been altered. Some programs track compression values. Each time an image gets saved, it often gets compressed. But if part of a file is changed, that part won’t have been compressed as many times as the rest. This page explains further.

But this method of analysis might not work on a really good Photoshop job. And the process, therefore, still requires human examination.

In a case such as this, you would definitely want to hire an image and video forensics professional to use their sharp eyes.

Future Developments

Video forensics fake images

But what if you have a ton of pictures and video to go through? What then? There’s not much you can do now, except go through every picture individually, which could take forever. But people are working on a way to fix this problem.

Under a program funded by the Defense Advanced Research Projects Agency (for $4.4 million), researchers from the U.S as well as Brazil and Italy are trying to invent a machine-learning algorithm to help. This algorithm will hopefully be able to sift through loads of images and video forensic data and comb out the false from the real.

How could that help with your court cases? Well, hopefully, once this software is developed, it can be applied to all sorts of areas. It could be used to analyze all the images on someone’s computer or on a public Facebook account for evidence in a case.

Maybe someday this software will come to a video forensics specialist near you. But for now, you’ll just have to rely on their human eyes.

Referenced from: U.S. Eyes Tools to Spot Faked Images, Video

Posted in Enhancement of audio and video | Tagged , , , , , , , , , , , , , , , , , , | Leave a comment

Judge’s Shocking Ruling on Hidden Audio Surveillance

Share on FacebookTweet about this on TwitterShare on Google+
audio surveillance

Wiretapping circa a long time ago

Judge’s Shocking Ruling on Hidden Audio Surveillance 

The NCAVF article titled “FBI Plants Audio Surveillance Outside Courthouse“ was originally posted on 05/12/2016. This is an update to that article based on current events and the judge’s ruling on evidence and the use of audio surveillance devices in the case. Please click the above link to read the original article.

US v. Joseph J. Giraudo, et al.

After hearing arguments from both sides, U.S. District Judge Charles Breyer found that federal investigators had “utterly failed to justify” their warrantless use of recording devices. In his 19 page decision, Judge Breyer wrote, “Even putting aside the sensitive nature of the location here, defendants have established that they believed their conversations were private and that they took reasonable steps to thwart eavesdroppers.”

The judge’s decision could set a significant precedent in regards to the use of hidden audio surveillance devices. According to Judge Breyer, even though a conversation is being held in public, where bystanders not party to the conversation may hear, none the less, the conversation is (at least in this case) considered private. Therefore, in order for these recordings to be used as evidence in this case, the FBI should have obtained a warrant.

At almost the same time Breyer was hearing arguments on the San Mateo case, U.S. District Chief Judge Phyllis Hamilton heard an almost identical case involving audio recordings that were also secretly recorded by the FBI, also without warrants, also right outside courthouses, and also involving real estate investors whom the FBI believed were bid rigging. This case originated just across the bay, in Alameda and Contra Costa counties, but Judge Hamilton ruled completely differently that Judge Breyer.

US v. Michael Marr, et al.

In her case, Judge Hamilton ruled that the audio evidence would be admissible. In her decision, Hamilton stated, “Having listened to the recordings at issue, the court finds that defendants did not take steps to protect the privacy of the conversations that were audibly recorded.”

Just like in Breyer’s case, attorneys for the defendants (in Hamilton’s case) argued that despite being outdoors, clients frequently have private conversations with their attorneys. Judge Hamilton, however, dismissed this argument, stating, “It has been the court’s observation that conversations near the courthouse entrance are frequently overheard by unintended and unseen listeners, even from inside the courthouse.” Hamilton, therefore ruled there was no expectation of privacy.

Have Your Cake and Hide it Too

Despite siding with the prosecution in ruling the audio recordings admissible, Judge Hamilton, none the less, also indicated, “it is at the very least unsettling that the government would plant listening devices on the courthouse steps.” hamilton’s statements are concerning. On the one hand, Hamilton says the recordings are fine, but on the other hand she admonishes the FBI agents for planting audio surveillance devices outside a courthouse.

In her decision, Judge Hamilton pointed out that part of the reason she ruled in favor of the prosecution is because the defendants , “…did not take steps to protect conversations.” Even in public, most people assume that a conversation between two people is considered private. That a nearby bystander is listening is irrelevant because that bystander is not a party to the conversation. The judge, however, seems to feel different, as her ruling reflected.

audio surveillance

The first device that was wiretapped

Same But Not Equal

There is a very stark and dramatic difference of opinion in these almost two identical cases: Judge Breyer ruled the evidence out by saying the conversations were clearly private; Judge Hamilton ruled the recordings in by saying the conversation were clearly not private. Yet in both cases, microphones were hidden in public areas just outside courthouses. This difference of opinion between two federal judges touches on an interesting idea: are wiretapping laws outdated? A wiretap may be defined as, “ the monitoring of telephone and Internet conversations by a third party, often by covert means.” This would include setting up a hidden audio surveillance device such as a microphone outside a courthouse. The word “wiretap” comes from the 1890’s when the actual phone’s wire was “tapped” by plugging into it so it could be listed to or recorded by the third party. This language shows just how outdated these laws may be since so much of today’s “wiretapping” is done wirelessly.

Need for New Laws Regarding Audio Surveillance

Both of these cases, however, seem to address a subject that affects everyone (not just these defendants). Recently, the government has been placing microphones and recording conversations on city buses, trains, and other locations, seemingly without the consent of riders. Activities like this would appear to be a clear violation of our 4th Amendment rights. So what may be the future of wiretapping laws? When the authors of these wiretapping laws first created them, they did not foresee today’s smartphones, button cameras, and microscopic digital recorders. Is it time to reconsider what constitutes wiretapping in the 21st century?

Posted in Enhancement of audio and video | Tagged , , , , , , , , , , , , , , , , , , , , , , , , , , , , , | Leave a comment

Can You Use a Selfie to Find Out a Kid’s Age?

Share on FacebookTweet about this on TwitterShare on Google+

Can You Use a Selfie to Find Out a Kid’s Age?

Forensic Image Analysis

Selfies can be used to identify age of adolescents

Did you know you can accurately guess the age of a child from the diameter of their pupils? Pupil diameter seems to increase pretty predictably as a child grows, unlike other features of the face, according to Neeka Parker’s assessment of pertinent studies in her paper “Age Estimation of Adolescents Using Eye Measurements from Various Angles in Videos.”

Did you also know that kid’s love selfies? I probably don’t have to tell you that a selfie is a picture a person takes of him or herself. Because of this, the selfie is often at a unique angle from other pictures. Selfies, though, are almost epidemic, especially among younger people and millennials. (Over 55% of millennials say they’ve taken a selfie.) Forensic analysis of selfies as regards to a young person’s age could be an important part of a legal case.

But can you figure out a child’s age from their pupils in a selfie? In her study, Neeka Parker wanted to find out what selfie angles and cellphone pictures did to the angle of an adolescent face and to how well you could see their pupils.

Participants

With permission from their parents, Parker gathered twenty-one children from ages eleven to nineteen, twelve girls and nine boys, some African-American and some Caucasian, with one Mayan. Most of these children had brown eyes, which will be important later.

Variables

With these participants gathered, Parker had to deal with some variables in the way eyes change. Pupils can be affected by lighting, medication, or even just a person’s mood. Parker asked her participants questions about their mood and medication so she could account for those variables. And, of course, she made sure the lighting was controlled.

Process

Parker took pictures with an iPad and a Nikon D3100 camera and video with the Nikon. She took reference pictures and then had the participants take selfies, which were analyzed in Photoshop.

Conclusion

Selfie Forensics
In the end, Parker found that since most of her participants had brown eyes, generally she could only distinguish the pupil on the Nikon images but not the iPad images or the Nikon video stills. This made further analysis and measurements difficult. Note that the pupil diameter is the most important part of judging a child’s age by their face and that brown is the most common eye color.

Thoughts

Although Parker did compare her device, the iPad, to other smartphones and concluded that camera hasn’t improved, at least in the iPhone, it would have been useful if Parker had included more imaging devices, including actual cellphones in her study. In my experience, a teenager would rather be caught dead than using something clunky like a tablet or iPad to take pictures. Selfies are the domain of the smartphone, which is light and convenient to use when taking pictures with your arm stretched awkwardly.

What does this mean for determining children’s age in the forensic image analysis of selfies? Parker says more research needs to be done. But right now it looks like whether your selfie is useful depends on what device it was taken with. She also pointed out it’s possible other ways besides pupil diameter may be needed to judge age in a selfie. But as phone cameras get better and better, it’ll get easier to figure out a child’s age and tons of other useful information, from the humble selfie.

Want to know more about forensic image analysis of selfies? You can read Parker’s research paper here.

Posted in Enhancement of audio and video | Tagged , , , , , , , , , , , , , , , , , , , | Leave a comment

How Forensic Video Analysis Changed History

Share on FacebookTweet about this on TwitterShare on Google+

How Forensic Video Analysis Changed History

forensic video analysis

Forensic Video Analysis can be used to solve mysteries of the past.

Forensics is not always limited to drama in courts or television in the forensic video enhancement and analysis world. What’s another use for forensic video analysis? Recently, forensic video expert Michael Plaxton of Crystal Beach, Ontario, got to change history. Plaxton used forensic video analysis techniques to reveal the truth behind evidence from World War II and therefore correct a mistake that’s over 70 years old.

Although Plaxton primarily works with the Hamilton police in Ontario, Canada, he also sometimes dabbles in private consultations on the side. According to the Hamilton Spectator, Plaxton was contacted by a Lucky 8 TV in New York because they were making a documentary about the second world war.

From the Halls of Montezuma…

Plaxton gained notoriety in forensics as a video forensics expert in helping to solve the case of the murder of Tim Bosma in Canada. (Bosma was murdered in 2013 by two people that came to see a truck he was selling through an online ad.) Plaxton’s work in the murder case was a major contributor to the conviction of the two suspects accused of Bosma’s murder. But when Lucky 8 TV contacted Plaxton, they didn’t want him to help with any criminal case, or not even a civil case for that matter. Lucky 8 TV wanted Plaxton to analyze the famous photo, Raising of the Flag on Iwo Jima (see below). As it turns out, forensic video analysis can even be useful in unraveling the details of events from history from over 7 decades (and several wars) ago!

Remember the men in this iconic photo?

forensic video analysis

U.S. Marines of the 28th Regiment, 5th Division, raise the American flag atop Mt. Suribachi, Iwo Jima, on Feb. 23, 1945 during World War II. (AP Photo/Joe Rosenthal, File)

Most people probably don’t know these men’s names. But for many years, the U.S. military thought that they were Ira Hayes, Franklin Sousley, Michael Strank, John Bradley, Rene Gagnon, and Harlon Block, from left to right. The military had already misidentified one of the men before. Turns out, the military was wrong again.

Although this case wasn’t about forensic video enhancement, per se, the case instead utilized modern forensic video analysis techniques. Analyzing frame by frame, Plaxton compared multiple sources of video and image footage (evidence) from when this photo was taken with the photo itself. He analyzed the equipment the men in the photo had been wearing earlier, reasoning that because of the fighting earlier the men in the photo would have unique differences in their equipment.

What Plaxton found by analyzing the videos and images is what revealed the facts of the case. According to the National Post, the equipment that ended up telling a new story were a broken strap on a helmet, a rifle sling with an odd swivel, a bulging right pocket, and a strip around one of the Marine’s neck.

The Conclusion

From the tiny details of the Marine’s equipment discovered by analyzing the historical videos and photographs, not only did Plaxton discover that one man, Franklin Sousley (second man from the left, foreground), was actually John Bradley, but he also found that John Bradley wasn’t even in the photo. The man who everyone had thought was John Bradley was in fact a man named Harold Schultz (third man from the left, foreground).

Forensic Video Analysis

By scrutinizing all the photographic and video evidence, it was determined that John Bradley had helped raise a flag earlier in the day, but he hadn’t been there for the iconic photo, seen above, which took place at a second flag raising later in the day.

This case is not only a great example of another use for forensic video analysis, but it’s also a great example of how the science of forensic video analysis isn’t just about forensic video enhancement. A good video forensic expert is also a detective with sharp eyes, a quick brain and a tenacious willingness to scrutinize every single frame of a video.

For more information on this case and the flag raising, watch the Smithsonian Channel’s The Unknown Flag Raiser of Iwo Jima, which debuted July 3rd, 2016.

Posted in Enhancement of audio and video, In The News - Surveillance | Tagged , , , , , , , , , , , , , , , , , , , , | Leave a comment

Surveillance Video Evidence Captures Intruder

Share on FacebookTweet about this on TwitterShare on Google+

Surveillance Video Evidence Captures Intruder

video evidence

Home Surveillance Captures Video Evidence of Pint-Sized Intruder

Safety is a major concern these days. Everywhere, folks are concerned about their personal safety from an attack, their business’s safety from theft or damage or even their home’s safety from burglary or intrusion. Video surveillance is an affordable and convenient first step in helping to establish a layer of protection. So when someone’s video surveillance is successful in capturing an intruder, what should one do with the video evidence? One home invasion “victim” invited the perpetrator over to visit again any time.

Intruder Caught on Surveillance Camera

Recently in Louisiana, a homeowner was reviewing the surveillance footage from her home’s security system when she noticed something both strange and unexpected: an intruder had entered her garage. This, however, was not a dangerous felon looking for an easy “score”, or worse. The intruder was a little boy. The pint-sized burglar ran into the garage, hugged the homeowner’s dog (attention that the dog seemed to thoroughly enjoy), and ran away before being caught. It turns out, however, that he was caught, caught by the owner’s surveillance system.

A Harsh Sentence

Typically, the whole point of a home surveillance system is to prevent break-ins, by acting as a deterrent, and to create video evidence of the intrusion in order to catch and convict the offender. In this case, the homeowner took decisive action: she wanted to invite the intruder over! She, however, was unsure the identity of the boy. The young prowler wore a baseball cap and avoided looking at the camera, thus protecting his face and identity! Typically, a video forensic expert could be utilized to enhance the video to try and discern the identity of the invader. In this case, however, it seems that the homeowner did not consider her intruder a threat and instead wanted him to know he was welcome to play with their dog any time.

In place of using video enhancement techniques, the homeowner turned to a slightly lower-tech method of identification: social media. While posting a copy of the surveillance video evidence on Youtube, the homeowner made known that she would like to find the identity of the trespasser. Her goal was not to get the young burglar in trouble, but rather to let him know he was invited to come play with her dog any time. After over a million views, she learned that the boy was indeed her next door neighbor. The boy’s family had a dog that passed away the previous year and he was in need of some four-legged affection. The boy’s family plans on getting a new pooch soon, but in the meantime, he now has regular play dates with the neighbor dog.

 

Posted in Enhancement of audio and video, In The News - Surveillance | Tagged , , , , , , , , , , , , , , , , , , , , , , , , , | Leave a comment