The question of fate is prominent in the Safdie Brothers’ life and work. Their propulsive new crime thriller, Good Time, stars Robert Patinson as Connie, a small-time crook on a mission through New York’s grimy underbelly to rescue his mentally challenged brother (Benny Safdie) from the grips of New York City’s stifling criminal justice system. But Connie and his brother’s fates are already sealed after the movie’s opening frames. In this movie, as in reality, people like them seldom win.

But as much as Good Time uses fate as its central villain, it probably wouldn’t have been made without some kind of preordained magic intervening. For nearly a decade, Josh and Benny Safdie have been making messy and immersive movies about the people that New York City left behind—the deadbeat dad in Daddy Longlegs or the lovesick junkie in Heaven Knows What—and were seen by some as the misfit saviors of a kind of visceral American new wave that died with John Cassavetes.

Their films have such a striking visual swagger, that when Robert Pattinson came across a still from Heaven Knows What, he knew exactly what to do. After a shot-in-the-dark email and a follow-up in-person meeting, Pattinson became a newly converted Safdie disciple. The brothers shelved their long-gestating project Uncut Gems—which they begin shooting next year with Jonah Hill attached to star—and wrote Good Time specifically for Pattinson. After a breakneck guerilla shoot through the streets of their hometown, the Safdies eventually found themselves on the steps of the Croisette, the toast of Cannes.

We spoke to them about making the transition from fringe to Hollywood, working with someone as famous as Robert Pattinson and of course, the dog dick controversy heard round the world.

So let’s start with the Robert Pattinson-dog controversy. How did you guys find out that he’d gone on national television and mentioned that you asked him to jerk off a dog while filming?
JOSH SAFDIE: I was watching it live and I thought it was hilarious. I thought ‘Wow Rob is really going there with story.’ I don’t even know where it came from. Then When I woke up in the morning and saw the PETA was investigating I thought it was even funnier. But when it got anti-Semitic and people were like ‘Look at what the two heathen Jews have done to our goyim,’ that’s when it started getting ugly. It’s so ridiculous. The reality is I didn’t ask Rob to give a dog a handjob, but he actually gave a parrot a rimjob.

Did Robert text you to apologize?
JOSH: I sent him a screengrab of somebody who tweeted something really nasty at me and I put a laughing emoji and he was like ‘Oh my god I’m so sorry, are you angry with me?’ And I was like, ‘No I don’t care!’ The only thing I didn’t want was for the movie to become ‘The Dog Dick Movie’.

Now that people know who you are, do you worry that you have to be more careful with what you say and do?
JOSH: We didn’t do anything! Nothing happened and this story was born out of nowhere. But then you get a view into the cycle of why people click on things and what they want to see. But that’s not on us.

It kind of gives you a window into what Robert has to deal with on a daily basis. What was it like being in his orbit? Did you ever feel sorry for him?
BENNY SAFDIE: It’s kind of insane that he has that ability. He needs to be aware all the time. In a way that did inform the character of Connie because here’s this guy on the run who’s hyper aware of where he is and what he looks like and where he’s going to go. That element of Rob’s personality really spoke to Josh for the writing process. Here was this guy who was kind of living the life of a guy on the run except he’s very famous and people recognize him. But he’s always trying to escape.

Do you think that extreme level of fame is toxic?
JOSH: Not really. Sometimes it can be overbearing but it’s a matter of how you deal with it. I don’t think it’s a blanket statement.

Did you guys have any trepidations about working with a star of his magnitude or was there a sense that having him in your movie will help get it funded and seen?
BENNY: The one thing we were very aware of was we didn’t want him to be recognized while we were shooting. We didn’t want to disrupt the world we were creating, so we were very aware of trying to stay ahead of the people who were tracking us. It was an achievement for us when we saw a tweet saying that we were in Adventureland 10 days after we left Adventureland. We were always moving really fast and were aware of the people who were trying to find us. But for the most part nobody recognized him or suspected anything because we weren’t presenting ourselves as a movie set on the streets. We were presenting ourselves like a construction site.

You guys are used to working with non-actors and people you find on the streets. How did you approach working with someone like Rob? Did he require a lot of direction?
JOSH: Part of the challenge of working with someone who’s appeared in a lot of films and comes with a lot of baggage is you can’t ask the person to play themselves like you can with a first time actor, where you just play with the performance and mold it to the movie. With someone like Rob you have to do a lot of work to establish him as a first time actor. So we spent a lot of time developing a character background that was meticulous in the way it spanned every three months of his life up until the moment the movie starts. We had to link him up with a lot of people who Connie might have been friends with, people who Connie would hate, people he would love, people he did time with, people who were Connie in another world. We went on weird walking tours of local spots in Long Island, Benny and Rob hung out in a garage up in Yonkers for eight hours, we went to a Dunkin Donuts. Basically a lot of real life exercises that raised the stakes of performance for Rob. A failure in performance doesn’t result in a bad review. It results in an embarrassment where someone is looking at you and asking why you’re pretending to be someone you’re not.

I know you shot a lot around the city without any permits. Talk to me about how you navigate something like that?
BENNY: We had permission and permits for a lot of places and once you do get permission for one spot, you do have a little bit of leeway along the streets. New York is actually pretty good about allowing you to shoot. They just want to make sure you have insurance. That’s the biggest thing. If you mess something up they want to make sure you want to be able to pay for it. Our location manager was pretty amazing in making it so that we could shoot in places like an active ER.

How much of the world depicted in your film—that kind of seedy underbelly of New York populated by amateur crooks—how much of that was part your world growing up?
JOSH: Since I was a teenager I’ve always been attracted to that type of a person. I’ve gotten into trouble with people like that. It’s just the type of person I’m attracted to for some weird reason, I don’t know why. I have a lot of different circles that I run around in and that’s one of them. These people are total stars in my opinion and in their own eyes too. Norman Mailer was a really big inspiration for this movie. I was reading The Executioner’s Song for the first time and in The Belly of The Beast and his writings about the American criminal and his fears about where America was heading, so there was a lot of research that was done in addition to personal interest and attraction.

Contemporary New York movies are often very polished and tend to show a very sterilized version of New York. Was your intention to resurrect or recapture the gritty New York movies in the vein of vintage Scorsese and Cassavetes?
BENNY: We shot the movie today on locations that are here now and so it’s just a matter of looking at the city in a different way. Instead of looking at it with your head up at all the big skyscrapers, you’re looking at it with your head down at the sidewalk, five feet in front of you. Because we’re not treating New York with that kind of awe, we’re able to see what’s actually still here. Those movies were made with the same kind of feeling. So if you make your movie with the same feeling and the same propulsion you’re going to get the same thing even though it is ‘now’. Yes there are cell phones and LED screens and flat screens, but it’s strange how you can show modern life and have it feel like it isn’t modern because people aren’t used to seeing it that way.

Are you bummed by the way New York has evolved?
JOSH: Every movie we shoot we’re worried that locations and business are going to become hit by our kiss of death because a lot of our locations have either been torn down or renovated beyond recognition. But I think New York will never be fully dead. Yes in a lot of ways the life is being sucked out of it, but I do think that the city will always find a way to cycle itself through. When the crash happens next it’s going to be very interesting to see what happens to all these buildings.

BENNY: Look at Times Square. It’s a cyclical neighborhood. It will always find its life no matter what. Everyone thought it was totally dead because of its Disneyfication, now look at it.

JOSH: They built a Dave and Busters that should’ve change the neighborhood but it’s kind of a dangerous and scary place

I found your depiction of Rikers particularly compelling and terrifying. Have you spent any time there?
JOSH: I’ve never been to Rikers but I’ve been through booking a handful of times and I have friends who’ve done time. I had Buddy (Durress) keep prison journals while he was doing time at a prison upstate and was also able to visit him when he was at Rikers. It’s assaultive in a lot of ways and especially on a sensory level. There’s a smell there that you’ll never forget. Everything is coated with bleach because they’re constantly wiping things down. Mailer writes about how in prison there’s a certain kind of person who wears it as a badge of honor and others who don’t want to talk about it, and how prison can separate the cowards from the really harsh. Rikers in particular is like that. They can never shut it down. There’s too many people there. Brooklyn House will become the next Rikers island. When you have a broken arm you can’t put a bandage on it. Shutting it down is inhumane. You have to look at the system at large if you really want to fix anything, but I’m not here to get on a soapbox. The depiction of Rikers in the movie is literally taken from Buddy’s journals. Everything from the race ratio to the violence. It’s all straight ripped right from reality. And we cast a lot of people who had recently gone through Rikers and a lot of the correction officers who were recently retired.

Let’s do a complete 180 from prison talk and discuss premiering your movie at Cannes. Did that feel like a breakout moment for you?
JOSH: The real revelation for us when Scorsese watched Heaven Knows What and then talked about us in a press release. That was a burning bush moment. But we had been to Cannes a couple of times at the director’s fortnight and it was a really great experience. But everytime we went we would always pass the red steps and look at them the way a Congressman looks at the White House. It was a very surreal experience and the regality of it almost made me think ‘Wait a second, I think we did make something here of value.’ At the same time there’s a long lineage of movies that I’ve seen from Cannes that I hadn’t liked. Cannes is an unbelievably scrutinizing place and people are very harsh on movies there but we made it out not only alive but thriving. It’s kind of cool.

Daniel Lopatin—who did the award-winning score for Good Time—told me that you guys took him to a party on Leo’s yacht. What was that night like?
BENNY: It wasn’t Leo’s yacht, it was one of his friend’s yachts. It was bizarre because our street caster—Eleonore Hendricks—was on the boat and she had an experience with Leo when she was 15. She was a model for Calvin Klein and she met DiCaprio at some event and gave him her number. She was 15 so she was living with her mom. He called her at like 3:30 in the morning and her mom answered. He said ‘Hey I’m looking for Eleonore, it’s Leo,’ and she woke her daughter up and said ‘Hey it’s Leo.’ Eleonore tells this to DiCaprio on the boat and it was just so weird. It was like we were there but we weren’t there.

You’ve said in the past that you see your movies as blockbusters but obviously you make them in a way that is very anti-Hollywood. As your profiles grow and the interest around your movies grows, do you worry that you’re going to lose that kind of DIY energy or do you plan on keeping your methods the same?
JOSH: No we’ll figure it out. We always find the pocket. We’re going to be who we are wherever we are.

BENNY: Independence is a state of mind. It isn’t a budgetary thing. You can make an independent film with $100 million dollars, it’s just a matter of how you approach it.

Your next film will be produced by Scott Rudin, one of Hollywood’s biggest producers, and, of course, Scorsese. Will they give you guys the freedom you need to realize your vision?
JOSH: I think that’s one of the most fun part about it. We’ve been making feature films for over a decade and while we were never really invited to the table but we always understood how it’s played. All of a sudden we’re being invited to the table.

BENNY: We’ve been working a long time. It’s not like we’re new on the block and getting starry-eyed over it.

JOSH: In a weird way people understand that we have nothing to lose because we know how to do this completely on our own. That allows us to be betting hard at the table because we know if it doesn’t work out, fuck it, we can do it on our own and we can do it on our own in a big way.

Do you think filmmaking is an exclusively intellectual pursuit meant for people who rabidly consume art and think critically? Or can anyone do it?
BENNY: Anybody can do it if they bring a unique perspective onto it. We’ve been to both Curacao and Lithuania and each time we go there we meet with all these young kids and we just ask them what they remembered most and they start telling us these incredible stories, and we say ‘Wow! You can make a movie about that,’ and they say ‘I can do that?’ If you really believe in it you just need to transfer the enthusiasm that you have telling us about it into the movie. This idea that you can do that was really new to them and exciting. Not everyone can do it, even though everyone has a camera. You do need a specific voice, but there isn’t an obstruction of budget that should deter you. If you have the desire and the will, you’ll make it happen.

Good Time opens in wide release this Friday.