Discussion: View Thread

  • 1.  Is the Academy review process broken? What to do?

    Posted 03-22-2009 12:20
    Hi all,

    This discussion gets to a problem I've observed in the leading
    business research in general. namely, that it tends to focus almost
    exclusively on empirical research. Of course, the title of Michael's
    paper seemed to be highlighting a cognitive function that should be in
    principle testable.

    But generally, I think the field needs to get beyond empiricism and
    into the a priori, to a greater degree. Otherwise, there can be very
    little discussion of theory. And ethical investigation, i.e., on the
    appropriate goals of business, will be hamstrung by the merely
    testable. But how does one test justice for example? Mostly, it's done
    by rational argumentation and thought experiements.

    There is clearly a lack of vision festering in business school
    curricula. And I dare say, this propensity is part of the problem.
    Until the ivory tower of business (and management in particular)
    ventures more into the philosophical, theory will continue to take a
    back seat to the empirical, and business students will continue to
    lack the interest and ability to consider foundational questions on
    the very nature of business that will make them truly wise and
    effective leaders.

    In case some of you missed it, there was an excellent article in the
    NYT recently outlining this problem as a failure in business schools
    partly responsible for the current economic crisis:
    http://www.nytimes.com/2009/03/15/business/15school.html?_r=1

    And the dealbook blog comment:
    http://dealbook.blogs.nytimes.com/2009/03/16/is-it-time-to-retrain-b-schools/?scp=3&sq=business%20school&st=cse

    My answer here is that until business schools begin training and
    hiring more professors interested in theory and rational argumentation
    (as opposed to a nearly exclusive emphasis on empirical study), and
    top journals begin publishing more of that kind of research, I'm
    afraid we'll continue to suffer a dearth of reviewers qualified to
    screen the more theoretical and philosophical papers. So to me, there
    is no quick and easy answer to this problem. Rather, it requires
    long-term cultural and institutional change.

    Best,
    Julian Friedland
    CU-Denver

    On Sun, Mar 22, 2009 at 9:34 AM, Barnett, Michael <mbarnett@coba.usf.edu> wrote:
    > Thanks for the interesting points, Amy.  I surely shouldn't make a habit of
    > responding to what I hope are many replies, but I'll respond to yours, and
    > then step back and, despite my urges otherwise, just observe responses for a
    > while.
    >
    > That's an important distinction you made about the purpose of the review
    > system.  I concentrated on selection in my note.  Perhaps the numerical
    > comments and notes to the editor items in the current process are of
    > sufficient caliber to achieve a good selection outcome.  I doubt it -- I
    > don't see how they'd be independent from comments (there are certain
    > "reviewer effects" here).  Also, comparison with the student rating process,
    > in my mind, only further highlights the flaws of the system.  A further
    > complication, though, is that there are other selection issues, such as best
    > paper proceedings, that may require qualitative insights.  But beyond this,
    > I think we should also view this as a development process, and we should
    > expect the reviews to provide meaningful and constructive insights to the
    > authors.  Maybe I'm wrong here -- perhaps this is something we can
    > sacrifice.  We've certainly come not to expect it.
    >
    > On the reviewer training angle, that's a good idea.  If you don't have
    > sufficient experience, perhaps you can "test in" to the reviewer pool by
    > attending a certain class (maybe online).  This may help somewhat, though I
    > think a good chunk of this remains lack of effort.  This might be where
    > Don's ideas on incentives would come in.
    >
    > Interesting discussion!
    >
    > Best,
    > Mike
    >
    > ********************
    > Michael L. Barnett, PhD
    > University of South Florida
    > College of Business Administration
    > Department of Management & Organization
    > 4202 E. Fowler Avenue, BSN 3527
    > Tampa, FL 33620-5500
    > Phone: 813-974-1727
    > Fax: 813-974-1734
    > Webpage: http://www.coba.usf.edu/barnett
    >
    > View my research on my SSRN Author page:
    > <http://ssrn.com/author=414796>
    >
    > ________________________________
    > From: Social Issues in Management Listserv on behalf of Amy Klemm Verbos
    > Sent: Sun 3/22/2009 11:06 AM
    > To: SIM@AOMLISTS.PACE.EDU
    > Subject: Re: [SIM] Is the Academy review process broken? What to do?
    >
    > Dear fellow SIMians:
    >
    > I respectfully disagree with Mike about (1) what is wrong with the review
    > process; and (2) solutions to the review process. Ultimately, the goal of
    > the review process is to accept quality papers for the program. Mike's paper
    > was accepted. The process failed if it should not have been accepted, and
    > succeeded if it was accepted.
    >
    > Imperfections in the process:
    > Need to assign X no. of papers to Y no. of reviewers. This means that there
    > will be imperfect matches between the expertise of a reviewer and the paper
    > that they are assigned to review. As we all have our own niches, there is a
    > need to review papers that are not a perfect match. However, I find that
    > even in that instance, it is pretty easy to differentiate good and bad
    > theory, research design, and writing.
    >
    > Assessing the quality of a review only by comments:
    > Reviewers rate papers. They don't all comment. Just as our student
    > evaluations are based on nos. but we tend to consider comments. And,
    > remember, reviewers can say confidential things to the session chair. Don't
    > condemn the entire system based on one aspect.
    >
    > Feedback isn't a process that comes naturally. Critical feedback is even
    > more difficult. My mother used to say: "If you can't say anything nice,
    > don't say anything at all." Obviously not a good way to review. But, even
    > with blind reviews, it is a problem. What to do? Training!
    >
    > Training of reviewers is spotty at best.  Brief reviews may be made by
    > inexperienced doctoral students who we URGE to participate by reviewing.
    > Should we stop? I think not. Encouraging early participation is a GOOD
    > thing.
    >
    > How else to learn to review than by doing? I have sought out opportunities
    > to learn to be a better reviewer. Provide PDWs--EVERY YEAR. Set up an online
    > training for reviewers at the SIM website. Improve quality through better
    > training!
    >
    > Hope this is helpful to the discussion.
    >
    > Amy
    >
    > -
    > Amy Klemm Verbos, J.D.
    > PhD Candidate & Adjunct Professor
    > Lubar School of Business
    > University of Wisconsin-Milwaukee
    >
    > -
    >
    > _______________________________________________________________________
    >
    > To send a message to the list, send your email to SIM@aomlists.pace.edu
    >
    > _______________________________________________________________________
    >
    > Visit the SIM Division website at: http://sim.aomonline.org
    > _______________________________________________________________________
    >
    > If you wish to unsubscribe from this list or change your delivery
    > options, you can do so online at:
    > http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1
    > _______________________________________________________________________
    >
    > _______________________________________________________________________
    >
    > To send a message to the list, send your email to SIM@aomlists.pace.edu
    >
    > _______________________________________________________________________
    >
    > Visit the SIM Division website at: http://sim.aomonline.org
    > _______________________________________________________________________
    >
    > If you wish to unsubscribe from this list or change your delivery options,
    > you can do so online at:
    > http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1
    > _______________________________________________________________________

    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org
    _______________________________________________________________________

    If you wish to unsubscribe from this list or change your delivery
    options, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1


  • 2.  Is the Academy review process broken? What to do?

    Posted 03-23-2009 16:58
    Dear Colleagues and Fellow SIMians:

    I would like to recommend to everyone, a series of articles that recently appeared in Organizational Research Methods. Art Bedeian and co-authors did a study examining the level of “scientific achievement” attained by members of the editorial boards of major management journals. Using a very objective approach, they found the recent levels of "scientific achievement" of members of editorial boards significantly lacking. This generated a series of commentaries by some high profile members of the Academy.

    From my perspective, I believe an interesting question is why is this the case? What is driving this phenomenon? I certainly have my own theory. I also think this is very much related to the editorial process for the AOM conference.

    With Warm Regards,

    Wayne Grossman
    Associate Professor
    Department of Business 3N219
    College of Staten Island/CUNY
    2800 Victory Blvd.
    Staten Island, NY 10314
    Phone: 718-982-2973
    Fax: 718-982-2965


    -----Original Message-----
    From: Linda Trevino LTrevino@psu.edu
    Sent 3/23/2009 3:54:12 PM
    To: SIM@AOMLISTS.PACE.EDU
    Subject: Re: [SIM] Is the Academy review process broken? What to do?



    Linda,

    Just a couple of thoughts.

    When you receive a submission that does not meet submission guidelines, you should quickly notify the Program Chair and return the paper unreviewed. Such a paper should not be entered in the review process.



    Also, if your assessment is that you are not competent to review a paper (without having to read massive amounts) you should again notify the Program Chair and return the paper.









    Linda K. Trevino

    Distinguished Professor of Organizational Behavior and Ethics

    Smeal College of Business

    402 Business Building

    Smeal College of Business

    The Pennsylvania State University

    University Park, PA 16802

    Phone: 814-865-2194 Fax: 814-863-7261

    Email: ltrevino@psu.edu

    From: Social Issues in Management Listserv [SIM@AOMLISTS.PACE.EDU] On Behalf Of Linda C. Rodriguez [blondie@pobox.com]
    Sent: Monday, March 23, 2009 9:51 AM
    To: SIM@AOMLISTS.PACE.EDU
    Subject: Re: [SIM] Is the Academy review process broken? What to do?









    I just quickly refreshed over the first three replies below. I have a question and its very nave. Im a first year professor and I know that my comments for reviews are not ever good enough, but then Im faced with two issues to overcome: 1) being a neophyte in the business and sometimes, I can barely understand some papers because of my lack of experience for which Im supposedly qualified (review mismatch) and 2) the theory is way over my head. So in order for me to make sense of a 40 page paper I have to read a lot of the lit review articles to get up to speed. Then, because its way past my level of expertise, how does one comment on improvements? On some of the reviews that I did early on, I got paper that did not meet the style guidelines. Those I kicked to the curb as automatically unreadable. 45 pages of 10 point font is inappropriate. But beyond basic formatting and grammar and style, some of the papers that Ive read for the Academy leave me cold because they are so long that I cant invest. I prefer to submit a five to ten page abstract or prospectus and then take comments for improvement of the final paper that will be submitted to a journal. In fact, the process is onerous to me that I prefer not to participate in the AoM for a while longer. In the tenure review process, getting a paper accepted at AoM is the same as getting a paper accepted at other smaller conferences as long as they are peer reviewed we arent too picky! To submit elsewhere does not mean that a paper lacks quality or contribution, and I think too many people feel that AoM is the only publication in town that has quality and contribution.



    No need to go through the headache of an AoM review unless I just want it for free and for fun to prepare for publication elsewhere. The process is just onerous either as an author or reviewer. Im in favor of 20 page papers that get to the point, but there is always one nay-sayer that asks why a particular topic wasnt asked or covered when, indeed, in the longer paper it was. Then that same visionless reviewer might not recommend a paper based on the fact that things arent coverednever asking if it could have been covered in the space allotted. Sigh, win-lose, all the way around.



    I have inflation-itis-

    My $.10

    Linda

    --









    From: Social Issues in Management Listserv [mailto:SIM@AOMLISTS.PACE.EDU]On Behalf Of Lee, Min Dong
    Sent: Monday, March 23, 2009 9:14 AM
    To: SIM@AOMLISTS.PACE.EDU
    Subject: Re: [SIM] Is the Academy review process broken? What to do?



    I am not an economist, but I do think Dons pricing model makes a sense if we can come up with pricing level that would not become a barrier for unfunded Ph.D. students (or maybe create an alternative system for students).

    Another suggestion I have is to reduce the length of papers to present only the essence of the paper. I was speaking with a friend in Physics recently, and he was astonished that our papers are often 40-50 pages long. His words: how can you possibly read all of them and reflect on them? Meaning if we spend too much time reading, we dont have enough time/energy/motivation to think. Or, if we are pressed against time, then we simply do not read the papers very carefully and come up with just perfunctory comments. As all of you know, good review takes time and good review on 40 pages takes very long time.

    There is another advantage of limiting paper length. I believe writing a good short paper that just contains the essence would be more difficult than writing a longer paper. And reviewing a very short paper for quality is probably easier, because authors cannot hide behind long quotes, lit. reviews and rhetoric etc. I tested this hypothesis with my class (student essays), and it seems to be trueJ So, the burden of proof falls on authors, not reviewers.

    I also think that since we will hear the presentations, we really do not need the full paper. Papers can be submitted to journals, not AOM conference.

    So, my suggestion is to limit the length of papers to something like 2000 words and allow reviewers more time to think.

    Thats my 2 cents.



    Paul





    Min-Dong Paul Lee, Assistant Professor

    Department of Management Organization

    College of Business

    University of South Florida

    4202 E. Flowler Ave., BSN 3403

    Tampa, FL 33620

    (813) 974-1721





    From: Social Issues in Management Listserv [mailto:SIM@AOMLISTS.PACE.EDU]On Behalf Of Donald S Siegel
    Sent: Sunday, March 22, 2009 10:49 AM
    To: SIM@AOMLISTS.PACE.EDU
    Subject: Re: [SIM] Is the Academy review process broken? What to do?



    HI Mike:

    Its always a pleasure to read your lucid and insightful comments on this list-serve. I understand your frustration. At the risk of sounding like a stereotypical economist, I view this as a pricing problem. With the proliferation of journals, increasing submissions to the AOM meetings, and greater pressure to publish (tenure committees are often focused more on quantity than quality), it is difficult to find reviewers. One solution is to raise the cost of submission, which is now zero, while simultaneously compensating reviewers (i.e., the revenue generated from submissions could be used to pay reviewers, with additional funds coming from the division). In other fields, especially in economics and finance, journals pay reviewers for on-time reviews or offer to donate a certain sum of money to charity. The latter option might be especially appealing to SIM members.

    Best regards,

    Don





    Dr. Donald S. Siegel

    Dean and Professor

    School of Business

    University at Albany, SUNY

    1400 Washington Avenue

    Albany, NY 12222

    Tel: (518) 442-4910

    DSiegel@uamail.albany.edu

    http://www.albany.edu/business/

    http://www.albany.edu/business/news_and_events/DonSiegel-CV.pdf

    http://econpapers.repec.org/RAS/psi32.htm

    http://ssrn.com/author=33607







    From: Social Issues in Management Listserv [mailto:SIM@AOMLISTS.PACE.EDU]On Behalf Of Barnett, Michael
    Sent: Sunday, March 22, 2009 10:29 AM
    To: SIM@AOMLISTS.PACE.EDU
    Subject: [SIM] Is the Academy review process broken? What to do?







    Dear fellow SIMians:



    I think the review process for the Academy of Management conference might be broken, within SIM and perhaps beyond. Id like to see if others share my concern, and if its not just me, Id like to push the division, or the Academy in general, to do something to fix it.



    This morning, I received notification that my paper was accepted at SIM. Thats good. Then I read my three reviews not good. The first reviewer offered a reasonable analysis and a couple good suggestions. It wasnt a journal quality review, but it was a solid review basically, of the caliber we should probably expect from a conference review, given time constraints, etc.



    Then theres the other two. Bearing in mind that the paper I submitted is a theory paper and has an intentionally edgy title designed to play off the existing literature and catch attention (Take a chit: Cognitive constraints on stakeholder response to corporate misconduct), heres the entirety of the sage feedback I received from the other two reviewers:



    R#2:

    Authors provide an interesting perspective on why CSR efforts may/often fall short. Paper would make an interesting conference discussion, and I would be interested to see the authors developing testable hypotheses from the propositions and collecting data.



    R#3:

    I have an issue with your title. I think it is inappproriate for an academic conference. I would leave off Taking a Chit.



    Include some up-to-date references (at least 2007 and above).



    So, R#2 appears unfamiliar with the concept of a theory paper, mistaking it for a research proposal (not that uncommon outside our world, but we should be getting reviews by people in our world), and R#3 focuses on the title (fine that was expected) and takes issue with references being from earlier than the prior year (this was submitted at the end of 2008; obviously written prior to that; also, I suspect I did have some post 2006 cites; and interestingly, R#1 rightfully suggests I add some cites from several decades prior).



    Two-thirds of my reviewers were disinterested, incompetent, or some combination thereof. Seriously its not that Im a prima donna author who is angered that others have not fully appreciated my written genius (maybe I am but thats not the point), but its that these reviews dont even address what I wrote.



    Beyond my anger that these two reviewers, for whatever it is worth to them, will be allowed to claim to have engaged in service to the profession, Im concerned about introducing this level of noise into the review process. How can the program chair make selections when the majority of data are at best useless, and at worst, wrong? Does this selection process lead to a program that is most useful to Academy members?



    This is not a slam at Barry Mitnick, the SIM division program chair. If anything, its a testament to his hard work and dedication, despite the troubled process. I cant imagine the headaches he has had, trying to sort information from noise, incompetence from earnestness, and so on, all on a grand scale and with limited time. Rather, its a call to change the process so that it becomes more useful, and perhaps efficient, if at all possible.



    So I think we first need to decide if the current model is broken in a way that cant be fixed. I think it is. The problem is that we cant obtain an adequate pool of competent and engaged reviewers. Therefore, we rely on the incompetent and disengaged, and to an increasing degree. This is a problem for our major journals (see the recent pleas for reviewers), and so its going to be even worse for SIM at Academy (cant even imagine the challenge at conferences with lower profiles). I dont think theres a way to get up to adequate quality while retaining or even increasing quantity. We could better screen potential reviewers (must have some basic qualifications dont think we do this now), and we could disallow bad reviewers from reviewing again (dont think we do this either), but we seem to need any semi-warm body we can find if we are to have the numbers.



    If we accept that the current system is broken and cant be fixed, then that frees us up to search for a new system. I hope the executive committee will do just this. What are some options? Most cynically, since the current system introduces a great deal of randomness, we might as well draw names from a hat 40% get spots. I seriously believe the outcome will approximate the current process in a few short years, as the reviewer pool further degrades under increased demand for reviews. If the result is the same, while the efficiency is much greater, we should do this.



    Another option is to decrease the number of reviews required for each paper. Two quality reviews would be better than three reviews where two are useless or wrong. This would allow for more stringent screening of reviewers, enabling us to leave out more of the incompetent and disengaged.



    A third option is to unblind the review process and allow selection, perhaps by a committee, based on record. This has many obvious chicken-egg problems how does one break in to the system? but it would increase the likelihood of securing a quality program. And if high status folks come to abuse their ability to get in by offering up bad papers, they could be barred from future seating based on record and instead have to go through a more intensive selection process (so this process could also degrade in time).



    A fourth option is to make all papers theme dependent. The committee could establish a set of topics the prior year, and then papers would need to fit with one of the themes, else it wouldnt be accepted. Reviewers would need only determine fit.



    Im sure there are more options, or even combinations. The things Ive suggested are off the top of my head. I hope we can put more of our heads together, so that we can do better than were doing now.



    Best,

    Mike









    ********************



    Michael L. Barnett, PhD



    University of South Florida



    College of Business Administration



    Department of Management Organization



    4202 E. Fowler Avenue, BSN 3527



    Tampa, FL 33620-5500



    Phone: 813-974-1727



    Fax: 813-974-1734



    Webpage:http://www.coba.usf.edu/barnett







    View my research on my SSRN Author page:
    http://ssrn.com/author=414796





    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org _______________________________________________________________________

    If you wish to unsubscribe from this list or change your delivery options, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=simA=1 _______________________________________________________________________

    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org _______________________________________________________________________

    If you wish to unsubscribe from this list or change your delivery options, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=simA=1 _______________________________________________________________________

    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org _______________________________________________________________________

    If you wish to unsubscribe from this list or change your delivery options, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=simA=1 ______________________________________________________________________________________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org _______________________________________________________________________

    If you wish to unsubscribe from this list or change your delivery options, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=simA=1 _______________________________________________________________________



    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org_______________________________________________________________________

    If you wish to unsubscribe from this list or change your deliveryoptions, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1_______________________________________________________________________




    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org
    _______________________________________________________________________

    If you wish to unsubscribe from this list or change your delivery
    options, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1


  • 3.  Is the Academy review process broken? What to do?

    Posted 03-23-2009 18:31
    That's an interesting reference, Wayne.  I'm not overly surprised that those researchers who serve on editorial boards tend not to have record-setting research records.  Of the pool of capable researchers who qualify for invitation, those who are completely research-focused are likely to refuse, and those who accept won't subsequently have as much time to produce research as those who do not.  We should all appreciate the efforts of researchers who are also willing to serve.

    Lori

    Lori Verstegen Ryan, Ph.D.
    Professor of Management
    Director, Corporate Governance Institute
    San Diego State University
    San Diego, CA   92182

    On 3/23/09 1:58 PM, "Wayne Grossman" <grossmanw@mail.csi.cuny.edu> wrote:
     
    > Dear Colleagues and Fellow SIMians:
    >
    > I would like to recommend to everyone, a series of articles that recently
    > appeared in Organizational Research Methods.  Art Bedeian and co-authors did a
    > study examining the level of "scientific achievement" attained by members of
    > the editorial boards of major management journals.  Using a very objective
    > approach, they found the recent levels of "scientific achievement" of members
    > of editorial boards significantly lacking.    This generated a series of
    > commentaries by some high profile members of the Academy.
    >
    > From my perspective, I believe an interesting question is why is this the
    > case?  What is driving this phenomenon?  I certainly have my own theory.  I
    > also think this is very much related to the editorial process for the AOM
    > conference.  
    >
    > With Warm Regards,
    >
    > Wayne Grossman

    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org _______________________________________________________________________

    If you wish to unsubscribe from this list or change your delivery options, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1 _______________________________________________________________________



  • 4.  Is the Academy review process broken? What to do?

    Posted 03-23-2009 20:20
    The key below is the word Recent. Many of those on boards now have chosen to contribute to the academy in ways other than recent having already establish a recorded of achievement less recently.
     
    Bob
     
    Robert Anderson Ph.D., CMA, Professor
    Paul J. Hill School of Business
    University of Regina
    Regina, SK
    Canada  S4S 0A2
    ph. 1-306-585-4728
    fax 1-306-585-5361


    >>> Lori Ryan <ljryan@MAIL.SDSU.EDU> 23/03/2009 4:31 pm >>>
    That's an interesting reference, Wayne.  I'm not overly surprised that those researchers who serve on editorial boards tend not to have record-setting research records.  Of the pool of capable researchers who qualify for invitation, those who are completely research-focused are likely to refuse, and those who accept won't subsequently have as much time to produce research as those who do not.  We should all appreciate the efforts of researchers who are also willing to serve.

    Lori

    Lori Verstegen Ryan, Ph.D.
    Professor of Management
    Director, Corporate Governance Institute
    San Diego State University
    San Diego, CA   92182

    On 3/23/09 1:58 PM, "Wayne Grossman" <grossmanw@mail.csi.cuny.edu> wrote:
     
    > Dear Colleagues and Fellow SIMians:
    >
    > I would like to recommend to everyone, a series of articles that recently
    > appeared in Organizational Research Methods.  Art Bedeian and co-authors did a
    > study examining the level of "scientific achievement" attained by members of
    > the editorial boards of major management journals.  Using a very objective
    > approach, they found the recent levels of "scientific achievement" of members
    > of editorial boards significantly lacking.    This generated a series of
    > commentaries by some high profile members of the Academy.
    >
    > From my perspective, I believe an interesting question is why is this the
    > case?  What is driving this phenomenon?  I certainly have my own theory.  I
    > also think this is very much related to the editorial process for the AOM
    > conference.  
    >
    > With Warm Regards,
    >
    > Wayne Grossman

    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org _______________________________________________________________________

    If you wish to unsubscribe from this list or change your delivery options, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1 _______________________________________________________________________

    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org _______________________________________________________________________

    If you wish to unsubscribe from this list or change your delivery options, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1 _______________________________________________________________________



  • 5.  Is the Academy review process broken? What to do?

    Posted 03-23-2009 21:34

    While Robert's point is well taken, as it was explained to me when I joined both the AMR and AMJ boards, my selection was based on as much how good a job I had done (e.g. depth, developmental nature etc.) as an ad hoc reviewer as much as my research reputation.  While much of the decision making work I have done had been published by the time I made it on to those two boards, it had not received the citation that it has more recently as the work has been "rediscovered."  Some level of publication success is necessary but for many of us, that was not the primary selection criterion.

     

    Regards

     

    Mark

     

     

    - - - - - - - - - - - - - - - - - - -

    Mark P. Sharfman, Ph.D.                               
    Professor of Strategic Management
    Director, Division of Management
    Price College of Business
    307 W. Brooks - Rm. 206A
    University of Oklahoma
    Norman, OK 73019-0450 USA
    +405.325.5689 (voice)
    +405.325.7688 (fax)
    Msharfman@ou.edu
    http://faculty-staff.ou.edu/S/Mark.P.Sharfman-1/

     


    From: Social Issues in Management Listserv [mailto:SIM@AOMLISTS.PACE.EDU] On Behalf Of Robert Anderson
    Sent: Monday, <st1:date year="2009" day="23" month="3" ls="trans" w:st="on">March 23, 2009</st1:date> <st1:time minute="20" hour="19" w:st="on">7:20 PM</st1:time>
    To: SIM@AOMLISTS.PACE.EDU
    Subject: Re: [SIM] Is the Academy review process broken? What to do?

     

    The key below is the word Recent. Many of those on boards now have chosen to contribute to the academy in ways other than recent having already establish a recorded of achievement less recently.

     

    Bob

     

    Robert Anderson Ph.D., CMA, Professor

    <st1:place w:st="on"><st1:placename w:st="on">Paul</st1:placename> <st1:placename w:st="on">J.</st1:placename> <st1:placetype w:st="on">Hill</st1:placetype> <st1:placetype w:st="on">School</st1:placetype></st1:place> of Business

    <st1:place w:st="on"><st1:placetype w:st="on">University</st1:placetype> of <st1:placename w:st="on">Regina</st1:placename></st1:place>

    <st1:place w:st="on"><st1:city w:st="on">Regina</st1:city>, <st1:state w:st="on">SK</st1:state></st1:place>

    <st1:country-region w:st="on"><st1:place w:st="on">Canada</st1:place></st1:country-region>  S4S 0A2

    ph. 1-306-585-4728

    fax 1-306-585-5361



    >>> Lori Ryan <ljryan@MAIL.SDSU.EDU> 23/03/2009 <st1:time minute="31" hour="16" w:st="on">4:31 pm</st1:time> >>>
    That's an interesting reference, <st1:city w:st="on"><st1:place w:st="on">Wayne</st1:place></st1:city>.  I'm not overly surprised that those researchers who serve on editorial boards tend not to have record-setting research records.  Of the pool of capable researchers who qualify for invitation, those who are completely research-focused are likely to refuse, and those who accept won't subsequently have as much time to produce research as those who do not.  We should all appreciate the efforts of researchers who are also willing to serve.

    Lori

    Lori Verstegen Ryan, Ph.D.
    Professor of Management
    Director, Corporate Governance Institute
    <st1:place w:st="on"><st1:placename w:st="on">San Diego</st1:placename> <st1:placetype w:st="on">State</st1:placetype> <st1:placetype w:st="on">University</st1:placetype></st1:place>
    <st1:place w:st="on"><st1:city w:st="on">San Diego</st1:city>, <st1:state w:st="on">CA</st1:state>   <st1:postalcode w:st="on">92182</st1:postalcode></st1:place>

    On <st1:date year="09" day="23" month="3" ls="trans" w:st="on">3/23/09</st1:date> <st1:time minute="58" hour="13" w:st="on">1:58 PM</st1:time>, "Wayne Grossman" <grossmanw@mail.csi.cuny.edu> wrote:
     
    > Dear Colleagues and Fellow SIMians:
    >
    > I would like to recommend to everyone, a series of articles that recently
    > appeared in Organizational Research Methods.  Art Bedeian and co-authors did a
    > study examining the level of "scientific achievement" attained by members of
    > the editorial boards of major management journals.  Using a very objective
    > approach, they found the recent levels of "scientific achievement" of members
    > of editorial boards significantly lacking.    This generated a series of
    > commentaries by some high profile members of the Academy.
    >
    > From my perspective, I believe an interesting question is why is this the
    > case?  What is driving this phenomenon?  I certainly have my own theory.  I
    > also think this is very much related to the editorial process for the AOM
    > conference.  
    >
    > With Warm Regards,
    >
    > Wayne Grossman

    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org _______________________________________________________________________

    If you wish to unsubscribe from this list or change your delivery options, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1 _______________________________________________________________________

    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org _______________________________________________________________________

    If you wish to unsubscribe from this list or change your delivery options, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1 _______________________________________________________________________

    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org _______________________________________________________________________

    If you wish to unsubscribe from this list or change your delivery options, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1 _______________________________________________________________________



  • 6.  Is the Academy review process broken? What to do?

    Posted 03-26-2009 11:20
    I’d like to respond to some of the comments made below. First, Mark (or Lori), for the reasons I describe below, I don’t think that this issue really pertains to your personal service on the editorial boards you list. Second, I think that some of the issues raised are more of a problem in some divisions than others. For example, the leadership of the SIM division seems to be drawn from a group of folks that have been around for awhile. In other divisions, “not so much (Borat, 2006).” Sometimes I am reminded of “Lord of the Flies.”

    Now, the claim is being made that an invitation to sit on major editorial boards is not just a question of one’s objective productivity, but their ability to execute competent reviews in a timely, professional and developmental manner. Thus, this would explain the lack of “scientific achievement” on major editorial boards. But, isn’t that the problem?? This discussion was initiated by Mike who, despite experiencing a positive outcome, had the integrity to speak out about the LACK of competence associated with the review process. Tim DeVinney said that in 20 years of participating in the conference review process he has noticed a decline in the quality of review. I have been studying Management at the doctoral level since 1991 and I also agree. However, I also believe that this is influencing what is appearing in major management journals.

    I think it might be useful to talk about the big picture with regards to Bedeian and colleagues’ articles. I don’t want to put words in their mouths, but the issue seemed to be one of peer review. It seems the overarching concern was that traditionally, the editorial process was based on a system of peer review. What I took away from the article was that, in Management, this system appears to be deteriorating. Obviously, they are not alone. Yes, of course, their measures, publication and citation counts, do not completely capture the construct of what is a “peer.” I would actually go as far as describing it as a somewhat narrow view of things. But, you know, its interesting that some folks are very quick to count publications and citations in some contexts, but in others, “not so much (Borat, 2006).” We might call this “contingent validity!” Feel free to use this “cultural learning (ibid.),” but please cite me!! I think Bedeian, et al. should be lauded. They had a perception about a social phenomenon, and attempted to back it up with evidence. It would not surprise me if they are well aware of the limitations of their measures.

    I have not read this article in a few months, but I seem to recall that there were people sitting on editorial boards with only 1 publication, and some had none! Quite a few if I recall. Therefore, the issue really is not one of putting individuals on editorial boards who do not have a “stellar” record. Forgive me if I am skeptical that journals such as AMJ, AMR, and JOM etc. are having a problem attracting qualified board members who have not served previously. I can understand (sort of) someone turning it down if they have served before. Honestly, where I come from, serving on an editorial board, without the experience of doing affirmative research, over a number of years, is kind of presumptuous and reaching. To me is seems very similar to the kind of culture that I know exists on Wall Street which has brought us to this state of affairs. I know of one instance in which an individual began serving as a so-called “gatekeeper” when he was an ABD! And, in his doctoral work, he had not even taken a Management course!! There are many such examples. Also please forgive me if I am not completely appreciative of this sort of thing.

    Finally, this becomes a real problem when it reaches the level of associate editor or, in some divisions, track chair at the national conference. And believe me, it has. This affects what we are seeing in journals. Many of us have entered this profession because we enjoy learning. But, track chair at a national conference or associate editor is not the place for an on-the-job training experience. At least it wasn’t in the past.


    With warm regards,

    Wayne Grossman
    Associate Professor
    Department of Business 3N219
    College of Staten Island/CUNY
    2800 Victory Blvd.
    Staten Island, NY 10314
    Phone: 718-982-2973
    Fax: 718-982-2965


    -----Original Message-----
    From: Sharfman, Mark P. msharfman@ou.edu
    Sent 3/23/2009 9:33:59 PM
    To: SIM@AOMLISTS.PACE.EDU
    Subject: Re: [SIM] Is the Academy review process broken? What to do?





    While Roberts point is well taken,as it was explained to me when I joined both the AMR and AMJ boards, myselection was based on as much how good a job I had done (e.g. depth,developmental nature etc.) as an ad hoc reviewer as much as my researchreputation. While much of the decision making work I have done had beenpublished by the time I made it on to those two boards, it had not received thecitation that it has more recently as the work has been rediscovered. Some level of publication success is necessary but for many of us, that was notthe primary selection criterion.



    Regards



    Mark







    - - - - - - - - - - - - - - - - - - -

    Mark P. Sharfman, Ph.D.
    Professor of Strategic Management
    Director, Division of Management
    Price College of Business
    307 W. Brooks - Rm. 206A
    University of Oklahoma
    Norman, OK 73019-0450 USA
    +405.325.5689 (voice)
    +405.325.7688 (fax)
    Msharfman@ou.edu
    http://faculty-staff.ou.edu/S/Mark.P.Sharfman-1/







    From: SocialIssues in Management Listserv [mailto:SIM@AOMLISTS.PACE.EDU] On Behalf Of Robert Anderson
    Sent: Monday, March 23, 2009 7:20 PM
    To: SIM@AOMLISTS.PACE.EDU
    Subject: Re: [SIM] Is the Academyreview process broken? What to do?





    The key below isthe word Recent. Many of those on boards now have chosen to contribute to theacademy in ways other than recent having already establish a recorded ofachievement less recently.







    Bob









    Robert AndersonPh.D., CMA, Professor



    Paul J. Hill School of Business



    University of Regina



    Regina, SK



    Canada S4S 0A2



    ph.1-306-585-4728



    fax1-306-585-5361



    robert.anderson@uregina



    Lori Ryan ljryan@MAIL.SDSU.EDU 23/03/2009 4:31 pm
    That's an interesting reference, Wayne. I'm not overly surprised thatthose researchers who serve on editorial boards tend not to have record-settingresearch records. Of the pool of capable researchers who qualify forinvitation, those who are completely research-focused are likely to refuse, andthose who accept won't subsequently have as much time to produce research as thosewho do not. We should all appreciate the efforts of researchers who arealso willing to serve.

    Lori

    Lori Verstegen Ryan, Ph.D.
    Professor of Management
    Director, Corporate Governance Institute
    San Diego State University
    San Diego, CA 92182

    On 3/23/091:58 PM, WayneGrossman grossmanw@mail.csi.cuny.eduwrote:

    Dear Colleagues and Fellow SIMians:

    I would like to recommend to everyone, a series of articles that recently
    appeared in Organizational Research Methods. Art Bedeian andco-authors did a
    study examining the level of scientific achievement attainedby members of
    the editorial boards of major management journals. Using a veryobjective
    approach, they found the recent levels of scientificachievement of members
    of editorial boards significantly lacking. Thisgenerated a series of
    commentaries by some high profile members of the Academy.

    From my perspective, I believe an interesting question is why is this the
    case? What is driving this phenomenon? I certainly have my owntheory. I
    also think this is very much related to the editorial process for the AOM
    conference.

    With Warm Regards,

    Wayne Grossman
    _______________________________________________________________________

    Tosend a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visitthe SIM Division website at: http://sim.aomonline.org_______________________________________________________________________

    Ifyou wish to unsubscribe from this list or change your delivery options, you cando so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=simA=1_______________________________________________________________________ _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org_______________________________________________________________________

    If you wish to unsubscribe from this list or change your deliveryoptions, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1_______________________________________________________________________



    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org_______________________________________________________________________

    If you wish to unsubscribe from this list or change your deliveryoptions, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1_______________________________________________________________________




    _______________________________________________________________________

    To send a message to the list, send your email to SIM@aomlists.pace.edu

    _______________________________________________________________________

    Visit the SIM Division website at: http://sim.aomonline.org
    _______________________________________________________________________

    If you wish to unsubscribe from this list or change your delivery
    options, you can do so online at: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=sim&A=1