Since you were all so helpful last week with my unknown flower, I remembered a plant I found a while ago. The fruit look a lot like chilis, I opened one up and the seeds are very similar as well.
Here are the leaves. I didn't get a photo of the flowers, but they were small and purple. If you know what this is, let me know :)
Edit: John identified the plant for me, it's Bittersweet, Solanum dulcamara, a member of the nightshade family.