[Systers-dev] Moving from Pickle Membership to Store in the Postgresql Database

[Malveeka Tewari]

It's highly possible that we just need a new table instead of an entire
database

I'll take a look at the wiki and then confirm

>From the pickle files, we need the user memberhship information - username,
email address for subscription, password, all lists that the user is member
of, user options and bounce info .


Thanks
Malveeka

On Sat, Jun 5, 2010 at 12:38 PM, Robin Jeffries <robin at jeffries.org> wrote:

> But we already have the postgres database for the dlists.
>
> First, what do you need that isn't already in that database? (I think that
> there is a diagram of the database tables on the systers wiki.  Anna made
> something last summer, and I think that's where it ended up)  You will
> certainly need to add information of your own, but that could just be an
> additional table in that database.
>
> What's in the pickle files that you need?  You don't want everything there,
> I assume.
>
> Second, if you do need a new database, we should start some communication
> among the archives folks (Pinar is probably your starting contact), the
> MM3.0 (Anna) and anyone else who is working on a database issue.  We will
> need to do these independently for the summer, but we ought to be talking to
> one another and understanding how to merge them, especially overlapping
> information.  We don't want to have to update a user email address in 4
> different places.
>
> Robin
>
> On Sat, Jun 5, 2010 at 12:31 PM, Malveeka Tewari <malveeka at gmail.com>wrote:
>
>> Hi Robin,
>>
>> For the single sign on project - we need to maintain the user membership
>> info (including passwords, mail delivery options etc. ) in a database that
>> can be accessed by the OpenID relay to enable easy password lookup for
>> single sign on.
>>
>> Currently in MM 2.x all this data is stored in pickle files as python
>> pickled objects. Storing this information in a separate db will make the
>> Single Sign On code more modular and extensble.
>>
>> For now, we have code working that when a new list and new members are
>> added
>> the use data gets stored into the membership_database. However, we haven't
>> written code to populate the database with information of existing users.
>> This is not a critical step - we can also do it after we get the OpenID
>> relay working for membership info for new members as well but this is
>> something that will eventually need to be done if want existing Systers
>> accounts to benefit from Single Sign On.
>>
>> We don't know of a good way to move the existing user info to the
>> membership
>> database, for now I've suggested Jaideep to write a python script that'll
>> parse the .pck files and populate the database.
>>
>> (We probably will not need this  once we move to MM 3 because from Anna's
>> discussions on MM - dev list,  I understand MM 3 will itself be storing
>> information in a database - however it's not clear to me how Syster's will
>> migrate from MM 2.x to MM 3 and so I've suggested Jaideep to get Single
>> Sign
>> on working for MM 2.x for now after storing user info in a separate
>> database. After that most of the development will involve getting OpenID
>> to
>> work with the membership database independent of the MM version)
>>
>> We could do with some help on how to do this more efficiently or pointers
>> to
>> existing code that already does that
>>
>> Thanks!
>> Malveeka
>>
>>
>>
>>
>>
>> On Sat, Jun 5, 2010 at 12:01 PM, Robin Jeffries <robin at jeffries.org>
>> wrote:
>>
>> > Can you tell us why you are doing this?
>> >
>> > Robin
>> >
>> > On Fri, Jun 4, 2010 at 11:46 PM, Jaideep Khandelwal <jdk2588 at gmail.com
>> > >wrote:
>> >
>> > > Hi,
>> > > I have been working for moving the User Membership from the Pickle
>> files
>> > to
>> > > the  Database in the Postgresql server, The mailman provides an option
>> of
>> > > dumping database with the use of the binary "dumpdb" that helps in
>> > storing
>> > > the data, now I seek some help in moving old list members information
>> > into
>> > > a
>> > > database . Is there any existing method for moving the Pickle data
>> file
>> > > into
>> > > an Database server.
>> > >
>> > > Thanks
>> > >
>> > > Regards
>> > > Jaideep
>> > >
>> > >
>> > > To unsubscribe from this conversation, send email to <
>> > > systers-dev+mentor+unsubscribe at systers.org<systers-dev%2Bmentor%2Bunsubscribe at systers.org>
>> <systers-dev%2Bmentor%2Bunsubscribe at systers.org<systers-dev%252Bmentor%252Bunsubscribe at systers.org>
>> >
>> > <systers-dev%2Bmentor%2Bunsubscribe at systers.org<systers-dev%252Bmentor%252Bunsubscribe at systers.org>
>> <systers-dev%252Bmentor%252Bunsubscribe at systers.org<systers-dev%25252Bmentor%25252Bunsubscribe at systers.org>
>> >
>>
>> > >>
>> > > or visit <
>> > >
>> http://systers.org/mailman/options/systers-dev?override=67&preference=0>
>> > > To contribute to this conversation, use your mailer's reply-all or
>> > > reply-group command or send your message to
>> > systers-dev+mentor at systers.org <systers-dev%2Bmentor at systers.org> <
>> systers-dev%2Bmentor at systers.org <systers-dev%252Bmentor at systers.org>><
>> > systers-dev%2Bmentor at systers.org <systers-dev%252Bmentor at systers.org> <
>> systers-dev%252Bmentor at systers.org <systers-dev%25252Bmentor at systers.org>
>> >>
>>
>> > > To start a new conversation, send email to <
>> systers-dev+new at systers.org <systers-dev%2Bnew at systers.org><
>> systers-dev%2Bnew at systers.org <systers-dev%252Bnew at systers.org>>
>> > <systers-dev%2Bnew at systers.org <systers-dev%252Bnew at systers.org> <
>> systers-dev%252Bnew at systers.org <systers-dev%25252Bnew at systers.org>>>
>>
>> > > >
>> > > To unsubscribe entirely from systers-dev, send email to <
>> > > systers-dev-request at systers.org> with subject unsubscribe.
>> > >
>> >
>> >
>> > To unsubscribe from this conversation, send email to <
>> > systers-dev+mentor+unsubscribe at systers.org<systers-dev%2Bmentor%2Bunsubscribe at systers.org>
>> <systers-dev%2Bmentor%2Bunsubscribe at systers.org<systers-dev%252Bmentor%252Bunsubscribe at systers.org>
>> >>
>> > or visit <
>> > http://systers.org/mailman/options/systers-dev?override=67&preference=0
>> >
>> > To contribute to this conversation, use your mailer's reply-all or
>> > reply-group command or send your message to
>> systers-dev+mentor at systers.org <systers-dev%2Bmentor at systers.org><
>> systers-dev%2Bmentor at systers.org <systers-dev%252Bmentor at systers.org>>
>> > To start a new conversation, send email to <systers-dev+new at systers.org<systers-dev%2Bnew at systers.org>
>> <systers-dev%2Bnew at systers.org <systers-dev%252Bnew at systers.org>>
>> > >
>> > To unsubscribe entirely from systers-dev, send email to <
>> > systers-dev-request at systers.org> with subject unsubscribe.
>> >
>>
>>
>> To unsubscribe from this conversation, send email to <
>> systers-dev+mentor+unsubscribe at systers.org<systers-dev%2Bmentor%2Bunsubscribe at systers.org>>
>> or visit <
>> http://systers.org/mailman/options/systers-dev?override=67&preference=0>
>> To contribute to this conversation, use your mailer's reply-all or
>> reply-group command or send your message to
>> systers-dev+mentor at systers.org <systers-dev%2Bmentor at systers.org>
>> To start a new conversation, send email to <systers-dev+new at systers.org<systers-dev%2Bnew at systers.org>
>> >
>> To unsubscribe entirely from systers-dev, send email to <
>> systers-dev-request at systers.org> with subject unsubscribe.
>>
>
>