Where did the notion that blacks can't be racist originate?

It seems whenever someone accuses individual blacks--or blacks in general--of racism, that charge is deflected by the assertion that blacks cannot be racist because they lack the power necessary to be racist. Where did this notion originate? Is there a seminal academic paper or journal article to which we can trace this idea? I've been trying to find its origin--so far without success. Thanks.