New Jersey Community Foundation: 
Commentary on the Neighborhood Leadership Initiative Rubric

Q&A between David Grant of the Dodge Foundation, and Ira Resnick and Faith Krueger, Directors of the Neighborhood Leadership Initiative, February 12, 2002, edited slightly for clarity. 

David: If technology is serving us well, there will be people who are on the Dodge website who have clicked on the whole Assessment section, read about some principles and concepts of Assessment, and then are saying, “Well, what would it look like?”  And the first model that we’d like to put on is your rubric. I’d like to begin by just asking some questions about it.  HOW did you create it?

Faith:  Ira and I sat down and actually worked backwards.  We took a look at some of the various projects that our NLI Fellows had done, what we considered to be some outstanding ones, and all the different components that went into that project – and then we also looked at ones that were not so great. 

David:  So, you were answering, “What does good work look like?” and, “What does not-so-good work look like?” based on past experience.

Faith:  Right.

David:  And what was the effect of asking these assessment questions?

Ira:  It seems the fundamental question that we’ve thought about, is “Exactly what do we do?”, “Exactly what is our business?” 

These were the two questions that I came away with.  It seemed that there were a number of cuts at this that we were kind of thrown by.  Are we in the business of providing workshops?  Is that what we do?  Is that what we should be evaluating?  How good the workshops are?  How good each NLI session is?  That was the first thing that comes to mind.  Because that’s what everyone always asks, right away.  Do people enjoy it?  Do they like the food?  Do they like the lighting?

Faith:  Was the space great?

Ira:  All that sort of stuff, since that’s what most people assess.  At the end of the workshop you pass out a flier and you know people check one to five and they get immediate feedback.  And what usually happens is that you find the workshop is really pretty good.

But you seem to be asking more challenging questions than that.  Which is annoying as hell.

David:  This is a great example of the question:  “Are you measuring what matters?”

Faith:  We’re hoping that food and heat and lighting don’t play anywhere into this at the end of the day.

Ira:   We asked, “Are we in the workshop business?”  That didn’t seem like the right thing to be evaluating, so what else do we do?  Well, then we thought, do we evaluate the project?  Do we look at the projects that people do and how GOOD they are?  Or how NOT GOOD they are?  And what we determined is that we’re probably not in a position from Morristown to really know if that project was right for Town A at the time that it happened. 

And even more than that, maybe that project was right for that person who did it two years ago.  An example being, maybe it wasn’t a great project from Morristown’s view, from our Ivory Tower, and yet, for that person’s development, at that particular time, it really led that person to do the following, three years later -- which is really our goal.  So, it seemed that that was another blind alley -- to evaluate the projects, sort of as thin as paper, without any depth to it. 

Well, then should we evaluate our teaching of leadership?  If that’s the basis of it, what principles are we giving to people that are portable, that they can use for Project A that we fund, and Project D, that we don’t fund? 

Well, we realized that would end up us evaluating ourselves, only, and so we pushed a little bit more, and then it got us to where we are now, which is that maybe we DO want to take a look at “What are the principles of leadership that we think are important, no matter what you do?” Said another way:  “What are the bedrock principles of leadership, no matter where you are, that might be important?” 

But that would again put the mirror back on us, are we really TEACHING those principles?  And that also would begin to put the spotlight back on our folks, are they LEARNING those principles? And the interplay between ourselves and them, the dialogue between teacher and learner – are we teaching the right thing? 

Oh, what a drag.  That would be HARD, because we’ve always wanted to DO that, as we realized it, but A) no one ever kicked us in the pants to do it, and B) once you do that, that means you’d have to realign the whole program to make sure each session begins to do that.  It fits into a larger master plan of where you want to go.  And that is really evaluating, or assessing as you might call it, or asking for feedback on a whole major program.  And yet, it was really time to do that, it felt right to sit down and take the time to do that. 

David:  It strikes me that the people who are holding your rubric in their hands, can read, can see, what you believe in.  They know when you say “leader” you mean something quite particular about it.  Sometimes when people are building a rubric they just use numbers to talk about the spectrum, 4, 3, 2, 1, but you are very clear that at your lowest level on your rubric here, you’ve used the word “alone.”

And then up to “circle of friends,” then to “cooperation” and then to “partnership” which must be at the heart of your vision for these projects and for the leaders that you are helping to create.

Sample section from Neighborhood Leadership Initiative Rubric:
Click here for full rubric.

	Alone	Circle of Friends	Collaboration	Community Building
Mobilizing a Leadership Team	Leader works alone without a wider leadership team.	Leader enlists advice or input on an ad hoc or informal basis from a network of 1-2 personal friends.	Leader establishes a team of 4-8 community partners who reflect some of the assets of the community and can provide support, guidance and advice.  Timeline for the leadership team is for the duration of this community project.	Leader establishes a leadership team that builds on the assets of the community  and ensures the project goals reflect the interests of diverse and significant community stakeholders.

Faith:  And really for future community initiatives that the leaders do.  It’s really a self-assessment – they can look at it at any given time and see exactly where they are.

David:  Yes, along the way.  It also strikes me that one thing that’s different about this rubric is that it’s essentially chronological. 

Ira:  It progresses both down on the left-hand column, as you’re suggesting -- steps of how you enter and engage in the community and then a progression across the page about personal skills.  Which makes it a little more dangerous, because we’re really putting ourselves out there. 

There is no question, people are going to say, “Well, I don’t agree with that.  I don’t agree with 4B, or 5D, or whatever.”  But this whole process made us say, “Look, if we’re not experts in community leadership, then we shouldn’t be allowed to run this program.”  If we’re not willing to put our values and our experience and expertise on the line, so people can critique OUR skills, then they shouldn’t entrust their learning to us.

David:  I’d like to underscore something as part of the case for thinking like an assessor first.  You’ve just said that essentially this assessment vehicle raises questions EARLY about core matters.  Whether people agree with you or not, these are questions that might have gone unframed or unexamined right through an entire program.

Ira:  Yes.  It’s a very challenging process.  We have to ask, “What am I doing that’s really the most important thing I’m doing?”

Faith:  It’s very reflective.

David:  Could I ask how you use this?  You’ve had the rubric now for a couple of years.  How do you use it with a new batch of leaders?

Ira:  After we wrote it, we pulled together about 25 of our leaders from past classes and passed it out and just said, “What do you think?”

David:  Good move.

Ira:  We told people “You’re coming to talk about a rubric that the Dodge people want us to do, and we’re submitting it, and what do you think about it?”  Basically, what it boiled down to was, they didn’t like it.  There was a real undercurrent – “Who are you to tell us what a good leader is?”

Faith:  They were very defensive.

Ira:  It was an audit, and people responded to it as an audit, as an evaluation. 

Faith:  It was not going well.

Ira:  So we just closed it down after about 20 minutes.  We learned a lot from doing that, so in 2001, we moved it up a few sessions and began to talk about it, to really ask people for feedback about the idea of feedback and assessment.  We really put it much more within its context.  It’s a tool from which you’ll understand maybe some larger contexts.  It’s not just a piece of paper to hand out.

David:  If I could underscore again what you’ve just said:  The point is not the rubric.

Ira:  Which we were making it when we first handed it out.  And even in 2000,  “The point was the rubric.”  By 2001, though, I think it was much more:  “The point is an assessment culture” and this is just a tool that one would use or NOT use.  And with the 2001 class in Trenton, the conversation was incredibly rich, and it turned out that the question that they had for us is this difference between cooperation and partnership, the question the rubric raises. 

One of the biggest problems for us in NLI has been to get our Fellows to go beyond their circle of friends to work with key stakeholders in their community with whom they really don’t identify.  The white guy who runs the bank, to be real specific.

It turns out the discussion that happened is that most people start their community programs with a bunch of friends.  But you really can’t hold your friends accountable for what they do.  And the trick is how do you work with a team, which is one of the items here, how do you hold the other people on your team accountable when they are your friends and they don’t do their job right?  And the conversation sort of moved into, “Wouldn’t it be nice to be working at a table with a group of people who aren’t your friends, but who are stakeholders or allies?” 

We’ve wanted to have this conversation for 5 years and we’ve never been able to get there.  And suddenly we’re talking about it because people see it on the rubric.  And THEY’RE saying it to us!  So then we said, “So you’re all saying to us that it would be smarter in the long run to start at the table with partners with whom you’ve done a one-on-one with, and not just getting your buddies to start this whole project.  And they say, ”Yeah, better to start with allies, better to do the one-on-one interviews that you talked about, and then get together allies at the table. 

Because it’s very confusing for us to work with friends and then make friends into partners.

That is, in a lot of ways, the nuts and bolts of what we try to teach in a whole year.  And we’ve never been able to get it.  We got it in one session because of the rubric.  The point is that it’s an objective, outside self-assessment tool that helps people think through for themselves, “How do I get to Step X?  How do I get to the Celebration?”  And it makes sense for people why it’s written this way.

David:  That was a nice example of planning backwards -- that vision of sitting around a table with people you have come to know.  You’ll never get there unless you plan backwards from that saying, “How do I meet them?  How do I first bring them together?  Under what circumstances?”

Faith:  The Circle of Friends is the same old, same old.  And to move into the Partnership, it eases a lot of the “stuff” that they’ve had to deal with and wondered why it hadn’t worked in the past. 

Ira:  We’ve done two things more up front this year than ever before.  One, talked about one-on-one’s.  How to do an interview with, essentially, a stranger.  So, we’ve done more role plays and practice on that.  And secondly, we redesigned the way we talk about and even what the homework assignment was to do an Asset Map of the assets in your community.  We broke it down much more so that people spend more time thinking about “Who are the people I’d like to know in my community, who I don’t now know?”  I must say, that has been a hard exercise for our people in previous years. 

For the last two years we gave a homework assignment and about 50% of the Fellows responded back.  This year 100% responded back with a new tool that we used to get them to do it. 

So, if you asked, “Are we teaching differently based on that conversation from last year, based on the rubric,” the answer is, “Yes, we are.” 

David:  The teacher in me would say the difference is that before, they said, “Here’s an assignment from my teacher.”  And this year what they saw is “Here’s a step that I have to take to get where I want to go, and it’s very clear to me where I’m heading, where I want to end up.”  So, I think that’s great.  Does that mean that you’re using the rubric as sort of a framework for conversation right off the bat?

Faith:  Yes.

Ira:  You mean with us, how we lead?  There’s no question.

David:  I like to say that rubrics should always say DRAFT on the top.  Have you been revising this one with each class as they use it and use it differently and embrace it more wholeheartedly?

Ira:  Well, I think we like what we did.

Faith:  The use of it has obviously stacked up more in it.  And also one other thing, the fact that vertically as well as horizontally it goes in stages.  NLI and the curriculum that Ira does is very foundational.  So, it just re-emphasizes over and over and over again, this foundational type of learning, and growth, into becoming leaders.

David:  To use the lingo – in the rubric now you have descriptors – have you ever thought of putting in indicators, you know, sort of specific stories, anecdotes?

Ira:  Yes.  As projects get better we think and could give examples because we have some stories now that we just didn’t have when we wrote this.  That’s a great thought.

David:  The boxes are infinitely expandable.

Ira:  That would be a lot of fun.

David:  Let me ask the bottom-line question.  Has the use of the assessment tool improved the projects?

Ira:  Yes.  There are examples now of people reaching out to stakeholders that they weren’t doing two years ago.  Major institutions - everywhere the people are beginning to work with the schools, the churches, the police departments, with banks.  This whole conversation that was part of the mix of our training, and they see the rubric now, you can where people are, when they’re doing their assessment survey, when they’re doing their asset map, when they’re doing their one-on-one, you can point to the different pieces in this rubric – that’s where they are. 

It’s done such a good job – it would be easy to make these boxes expandable because we’ve got so many examples.  We had NONE of that 4 years ago, just had NONE of that.  Now we have rich stories when we bring back Fellows to talk with each succeeding class.

Faith:  We were recently in Camden two weekends ago and we had some of our Fellows come back and give stories about how they dealt with their projects and how they networked and how they reached out to all the different organizations and agencies and faith-based institutions.  I know both Ira and I felt really good when we walked out of there because we felt we had really accomplished a lot of what we had wanted to do from changing the program from the Johnny Appleseed approach to one that kind of pushes you to form these partnerships that otherwise you may have stayed clear from.  For the Fellows that are now in the class, they can see that it is possible -- if you follow these steps and if you think this way and you kind of measure where you are along the way, there are several possibilities that can happen.

David:  I’m imagining that there may be people who are looking at this on the Dodge website, who admire what you have done, but their project will, of course, be different.  So, I’m wondering if you have any advice for them -- the sort of “thinking like an assessor” advice, independent of the specific content of your program.

Faith:  I think you have to be open-minded.  That unless you approach it in the right way people can be very defensive about it.  I think you need to be very much open-minded – I think you have to be willing to learn – to step – to expand, and to really be self-reflective, even before you go to a rubric and begin a rubric, because you really do have to examine, “How good are we?”  “Are we delivering now?”  “Where are we not doing the best job and where can we do better?”  And I think it’s also very self-reflective for us.  We got a lot out of it personally, doing this.

Ira:  Just to repeat, this challenge of “What is it that we really do?” is, I think, something that all of us need to spend a lot more time on.  The rubric is a good neutral way of raising that question with staff.  “Let’s think through this process we’re now engaged in:  What is it?  What’s our business really, ultimately?  This is a really good objective way without pointing fingers at anybody on the staff.  This is not, “You’re going to lose your job tomorrow.”  You created a nice culture about that at Dodge.  “You’re not going to lose your grant tomorrow, that’s not what this is about.  This is to make the work better.” 

It’s just a good opportunity, it seems to me, for the staff to kind of take some time out to look at “What is our business here?”  “How do we re-align our day so that we’re spending time on the things that really matter?”  Whether you use that or not with your constituents or your consumers, you know, your clients, to use a bad word, whether you use it or not, it seems to be a nice internal document to use in any event.  A nice objective way to do that.

David:  Great.  Now, of course you know I blew your chance to keep it an internal document by excerpting it in the Dodge Annual Report in 1999.  So my last question is whether you’ve had any book or movie offers, media attention, that sort of thing.

Ira:  Well, yeah.  The Annie E. Casey people in Denver called us up, and said, “We were really intrigued by your rubric, how did you do it? why are you doing it?”  And at least in some community foundation circles who run these sorts of leadership training programs, it has caused somewhat of a stir, that we have spent that much time on this.  And in some cases, people’s reaction is, ”Gee, that’s really interesting.”  And then some people’s reaction is, “Too much.  Too serious.” 

David:  Anything else you’d like to say before I turn off the tape recorder?

Faith:  You have to really be open-minded to do this exercise.  You have to be willing to examine yourself, your product, whatever it is that you do.  It is very self-reflective.

End of Interview