python - How to delete entities not found in feed on GAE -

-

I'm updating items from a feed on datastore 200 items at a time (which can be approximately 40000 items) Problems It is that the feed can change and some items can be removed from the feed. I have this code: 

  Class FeedEntry (db.Model): name = db.StringProperty (required = true) def updateFeed (offset, number = 200): response = fetchFeed (offset, Number) FeedItems = parseFeed (Feedback) feedEntriesToAdd = [] Items for items in feed: feedEntriesToAdd.append (FeedEntry (key_name = item.id, name = item.name)) db.put (feedEntriesToAdd)   
 How can I find out which items were not in the item feed and they were removed from the datastore? I thought about creating a list of objects (in datastore) and just remove all those items that I updated and wanted to delete.  
 PS: All items. IDs are unique to that feed item and are compatible.   
 
 
 If you add a DateTimeProperty with  auto_now = true  The last revised time of each unit will be recorded. Since you update every item in the feed, when you are finished, then you will come several times after the start of the moment, so any feed prior to any date is no longer there.  
 Javier's generation counter is equally good - we are guaranteed to grow only during a refresh, and there is never any deficiency during refreshment.  
 Not sure from the docs, but I hope the datetime property is bigger than an integer property, the latter is a 64 bit integer, so they  may  be of the same size It may be, or it may be that DateTimeProperty stores several integers suggesting that it is 8 unlike 10 bytes.  
 But remember that by adding an additional asset, which you query, you are adding another index, so the difference in the size of the field is diluted to the ratio of the overhead, besides some 40 bytes Folds are not more than $ 0.24 / g / month.  
 With any generation or date-time, you do not need to delete the data immediately, you can filter your other questions, latest refresh date / generation, which means that you can immediately Does not remove if the feed (or your parsing) becomes fun and fails to produce any item, or only produces a few, then the latest lie used as backup Maybe depends entirely on the application whether it is worth.

Comments

Post a Comment

Popular posts from this blog

mysql - BLOB/TEXT column 'value' used in key specification without a key length -

-
I have developed an extension which works up to 1.6 on Magren (I'm trying Enterprise Edition, And I think the community is the same problem, because it is the same code). In my install script, I see the $ installer-> gt; CreateEntityTables ($ this- & gt; getTable ('alphanum / info')); The installation is done until it is not in the _text unit table. It crashed there! It turns out that when I log in to SQL and run it via PHPmyadmin, then this error is: Blob / Text column 'value' is used without the key 'key' . I saw the code there, and this is what is trying to create an index on the value column: -> addIndex ($ this- & gt; getIdxName ($ eavTableName, array ( 'attribute_id array (' attribute_id ',' value ')) - & gt; addIndex ($ this- & gt; getIdxName ($ eavTableName, array (' entity_type_id ' , 'Value')), array ('entity_type_id', 'value')) If there is no if statement is n...
Read more

c# - Using Vici cool Storage with monodroid -

-
Oh, for the moment I am not sure how it is officially supported, but for the success of the people Using Monodroid Vic Cool Storage I am able to leave and compile assemblies in my project, but while trying to use some classes, the errors of time have to be gathered. Especially when trying to connect like the example for Mono Touch string dbName = path. Cobain (environment. Gatefolder path (environment specialfolder personal), "mydb.db3"); // The following line will be called coolestation, where is the database, // to make it if it does not exist, and call a delegate that creates the necessary tables (only when the database file was newly created //) CSConfig.SetDB (DbName, true, () => {CSDatabase.ExecuteNonQuery ("Subscriber Integer Primary Key Attribute, Name Text (50) No, Number, Date First Text (30)");); There is no difference when I try to use the methods of CSConfig and when I cscfif If I try to pass 3 aggregates to STDB (3) then I get the invalid ...
Read more

python - referencing a variable in another function? -

-
I am an experienced TCL developer and write my own processes to help myself. I call it a proc i callVar, print out the value of the variable in a certain format, so I know that it is variable and what is the value "set foo 1; putVar foo" result 'foo = "1" I want to do the same thing in Python, but I can not find the answer :( In TCL I use the up command because when I pass the name of the variable, then I can see the value at the top. . I understand that an advanced light in the dragon There is no mechanism of r, but with its introspection capability, I would think that this is something trivial, but I am not getting the answer anywhere. It seems that this should work: and expanded: def foo (): y = "Hello World" printwor ("y") foo () y = "Hello World " You can, but you should not do so if your code is correct If is needed , then there is something wrong with your design, and you should probably use the dictionar...
Read more