Grammatically, Chinese is extremely easy, with syntax very reminiscent of English, surprisingly. But it's the memorization of the character based system that is the most difficult, even to native speakers. And for foreigners the sounds can be difficult too.
Japanese imho from knowing both natively is more difficult because of its mixed historical background and need for conjugation..which doesn't exist in chinese. But the pronunciation is much easier.